Welcome to the Cisco Support Community Expert Lab session. I am Hitesh Kumar (CCIE SP #38757) and I work for the High Touch Technical Support (HTTS) team in Cisco. Along with me Rahul Rammanohar (CCIE R&S, SP #13015), Vinay Kumar (CCIE R&S  #35210) and Ruchir Jain (CCIE R&S #26911) will be active on this thread to discuss the challenges you run into while configuring mpls vpn setup. We will try our best to resolve your queries by providing supporting documents. If needed, we can setup a live interactive session to respond to the queries based on your inputs. Please provide your inputs/queries by replying to this discussion thread. Introduction & Topology Description: Many Service Providers use MPLS/VPN as a technology to provide connectivity between multiple sites for customers. We would like you to look through the topology and let us know what doubts you might have on configuring the various protocols/features, working of the various protocols in this scenario, manipulation of the traffic flows and the MPLS packet forwarding. The MPLS/VPN topology provided depicts a typical section of a Service Provider network. The routers in the cloud belong to the Service Provider and are used to provide MPLS/VPN services to customers. Customer routers (CE) connect to the edge routers (PE) of the Service Provider cloud. The Service Provider uses Route Reflectors (RR) to minimize the number of BGP neighborships within the core and they use 2 or more RRs to provide redundancy. In this scenario, a Service Provider is using 7 routers of which, 2 routers are RRs, 2 routers are Core routers (P) that do not connect to any customer and 3 PE routers that connect to the 4 different sites of a customer (say ABC). Customer ABC has 2 Hub sites (on the right side of the diagram) and 2 Spoke sites (on the left side of the diagram). The hub sites can talk to each other via the service provider link or via a backdoor link. The spoke routers cannot talk to each other directly, but need to talk via either of the hub sites only. The customer has configured OSPF as the IGP within his network and the Service Provider has configured ISIS. Additional Reference: How to configure MPLS VPN    ASK THE EXPERT : Introduction to MPLS VPN (archive) How to configure tag switching and MPLS ... View more

Community Tech-Talk series is designed to bring technical Experts from Cisco to share their insights on specific topics, selectively chosen based on most-common conversation themes on our technology area on community from our technology communities.   Session Objectives: Basic Architecture Overview Understanding Unicast & Multicast packet flow Troubleshooting Packet loss Issues. You can read the Blog ... View more

I am Hitesh Kumar (CCIE SP #38757) and I work for the High Touch Technical Support (HTTS) team, a team that provides reactive technical support to majority of Cisco’s premium customers. My team and I have been providing support to leading Service Providers and large Enterprise customers for many years. As majority of these customers are transitioning to XR platforms in their network, we wanted to discuss about Packet flow and Troubleshooting Packet Loss on the ASR 9000. Rahul Rammanohar (CCIE R&S, SP #13015), Vinay Kumar (CCIE R&S #35210) and myself have created this blog and a video discussing the same. Today all Service providers’ want to optimize operations, reduce expenses and improve scalability and flexibility to deliver Next-Generation Internet experiences. Keeping these requirements in mind Cisco came up with the ASR 9000 which is more mobile, more virtual and yet more simple to manage. We have covered the following topics in both the blog and the video: ASR9K building blocks. Packet Flow - Unicast & Multicast. Load Balancing on Fabric - Unicast & Multicast. Troubleshooting Packet Loss. Watch the Tech-Talk Video and you may download the presentation here: slides Overview The Cisco ASR 9000 Series was introduced in two form factors the Cisco ASR 9010 & ASR 9006. The ASR 9006 is a 6 slot chassis which can provide throughput up to 1.7 Terabits per second and ASR 9010 is a 10 slot chassis which has throughput up to 3.5 terabits per second. Cisco later introduced the ASR 9001 and the ASR 9922. The following table compares the various chassis. Models 9006 9010 9922 9001 Throughput 1.7 Tbps 3.5 Tbps 15 Tbps 120 Gbps Slots 4 slots for Line   cards 2 slots for RSP 8 slots for Line   cards 2 slots for RSP 20 slots for Line   cards 2 slots for RP 2 slots for MPAs Integrated RSP ASR 9000 runs next generation IOSXR software which is highly secure, scalable and modular network operating system which allows uninterrupted operation. Cisco ASR 9000 Series Routers is a fully distributed architecture which means all the forwarding decisions are taken by line card itself. RSP doesn't have any forwarding engine and it’s mainly responsible for control and management plane functions. RSP has a switch fabric and a route processor that are logically separated. The switch fabric provides non-blocking data connection among line cards and a connection between the line cards to the RSP for control packets. RSP comes in three flavors RSP2, RSP440 and ASR-9922-RP. The following table compares the three. Model RSP2 RSP440 ASR-9922-RP Processors PPC Dual Core 1.5GHz Intel Quad Core 2.27 GHz Intel Quad Core 2.27 GHz RAM RSP-4G: 4GB RSP-8G: 8GB RSP440-TR:6GB RSP440-SE:12GB -TR: 6GB -SE: 12GB Nv EOBC No Yes, 2 x 1G/10G SFP+ Yes, 2 x 1G/10G SFP+ Switch Fabric Bandwidth 92G + 92G (with dual RSP) 220G + 220G (with dual RSP) 660G + 110G (with dual RSP) Supported in Chassis 9006, 9010 9006, 9010 9001 (integrated RP) 9922 Each line card has its own CPU as powerful as the RP CPU that handles the control plane. There are multiple network processors that handle data forwarding and feature processing. There are 2 generations of line cards, the first generation uses the Trident Network Processor and the second generation uses the faster and more scalable Typhoon Network Processor. In the Video we talk about A9k-4T which is a generation1 line card and A9K-24x10G which is a generation 2 line card. The table lists the ASR 9000 cards based on the generation. Generation 1   (Trident Based) Generation 2   (Typhoon Based) One FIA A9K-40GE-L A9K-40GE-B A9K-40GE-E A9K-8T/4-L A9K-8T/4-B A9K-8T/4-E A9K-4T-L A9K-4T-B A9K-4T-E A9K-2T20GE-L A9K-2T20GE-B A9K-2T20GE-E Two FIAs A9K-8T-L A9K-8T-B A9K-8T-E A9K-16T/8-B A9K-2x100GE-TR A9K-2x100GE-SE A9K-24x10GE-TR A9K-24x10GE-SE A9K-36x10GE-TR A9K-36x10GE-SE A9K-MOD80-SE A9K-MOD80-TR A9K-MOD160-SE A9K-MOD160-TR Unicast Packet Flow   The ASR9k uses a two-stage fully distributed forwarding architecture. The local CPU on the line card creates software FIB table from the RIB and MPLS database information that it receives from the RSP CPU and locally available ARP information. Finally it programs the hardware FIB adjacency table in the NP for hardware forwarding. On the ingress line card (say a generation 1), NP does the first stage packet lookup. The NP is responsible for all L2/L3 forwarding lookups and feature processing like ACL and QOS. From the ingress lookup we get information about the egress NP and the frame is sent to Bridge ASIC. Bridge ASIC translates the header to a format that the Fabric Interface ASIC (FIA) can understand and forwards the frame to the FIA. FIA connects to the fabric and the frame is sent to the egress line card via the fabric. On the Egress line card (say a generation 2), the frame is received on the local Switch Fabric and forwards the frame to egress NP via the FIA. The egress NP performs the L2 rewrite and applies all the egress features like ACL and QOS on the frame and sends it on the wire via SFP+. Important thing to remember is that even in case of local switching, meaning if ingress and egress ports are on the same card, the frame has to traverse through Switch Fabric on the RP. Local Packet Transport System (LPTS) LPTS or Local Packet Transport Services is a very integral component of an IOS-XR system. It enables the router to act as a distributed system by having various protocol processes to be run across the RPs and the LCs. Packets that are destined to the router are forwarded to LPTS which is determined after a FIB lookup on the NP of the ingress LC. The LPTS Pre Internal FIB Table will be in line to make sure if the packet can be matched against any of the bindings. Depending on the match, action will be taken to either deliver the control packet to processes on the LC or on the RP. LPTS handles packets such as ICMP packets, Management plane traffic (such as Telnet, SSH, XML…) and Control plane traffic (such as OSPF, ISIS, BGP, IGMP, PIM, LDP…). Not all packets destined to the router will be subjected to LPTS, exceptions are packets such as LACP, CFD, E-OAM, CDP, ARP and BFD, Net flow sampled packets etc punt them directly to LC CPU. One of the biggest strengths with LPTS is the dynamic ACL creation which is configuration driven with no user intervention. Let us take BGP as an example, when you have no configuration, we would have a fixed policer handling traffic for TCP port 179 with any destination/source address which is heavily policed. As you configure and start to bring up a BGP session, an entry is dynamically created with the source address which is a local address and port 179 that is moderately policed. Once the session is established, another new entry is created for the L3 and L4 data that is lightly policed. LPTS also polices the incoming packets destined to the router. This policing mechanism is similar to Control Plane Processing on other platforms that are needed to protect critical applications. These policer rates are pre-defined for different flow types and are applied to the control and management plane traffic destined to the router. These policer rates can be configured by user for LPTS either per node or globally. Multicast Traffic Flow Multicast related processes such as PIM, IGMP and MRIB are running on the RSP CPU, hence the Multicast PIM and IGMP control packets are forwarded to the RSP CPU. The MRIB or (Multicast Routing Information Base) process is responsible to build the multicast RIB table based on the information from PIM and IGMP. The MRIB will download the table to the MFIB process on the LC CPU. This MFIB process will program the hardware tables on the NPs, Bridge ASICs and FIAs. These hardware tables which include the MFIB, the MGID (Multicast Group ID) and the FGID (Fabric Group ID) are used by the various ASICs for multicast replication and forwarding. Software switched packets are sent to the local line card CPU if the host is present on the same line card else the packet has to be forwarded via the switch fabric on the RP. For example (slide 7), we have one source S1 connected to NP0 on the Generation 1 line card and we have 5 receivers, one receiver on the same line card and four on the Generation 2 line card. The packet is received by NP0 which performs RPF check and determines the FGID and MGID. It then forwards the valid packets to the Switch fabric on the RSP (same process applies to L2 and L3 multicast). There are 4 places where the replication takes place, first being RSP switch fabric which replicates the packet to the appropriate Line Cards using the FGID. Second stage of the replication is on the fabric which replicates the packet to the corresponding FIAs. Third stage of replication happens on the FIA, it replicates the packets to the appropriate NPs based on the MGID. Final stage of the replication is on the NP, which replicates the packets to the corresponding interfaces. The egress NP also takes care of egress features such as ACL and QOS. The best part of replication on ASR9k is that it performs intelligent replication on all stages. Replication takes place only at places where it’s required. ASR9k Switch Fabric Path The switch fabrics are present on the RSPs and always remain in Active/Active mode, though the RSPs work in Active/Standby mode. This means the fabric on both RSPs is available for data forwarding hence Cisco recommends using dual RSPs. A generation 1 line card that has one FIA has two fabric links to each RSP. A link is of 23Gbps in each direction. So with one RSP we get 46 Gbps and with two RSPs we get 92 Gbps of bandwidth. A generation 1 line card that has two FIA has four fabric links to each RSP. A link is of 23Gbps in each direction. So with one RSP we get 92 Gbps and with two RSPs we get 184 Gbps of bandwidth. A generation 2 line card has four links to each RSP and each link is of 55 Gbps. So with dual RSPs we will have total of 440 Gbps bandwidth and with Single RSP we get 220 Gbps. In our example we are using RSP440s, hence a generation 2 line card will have a bandwidth of 55 Gbps for each of the 8 channels. If we use a generation 2 line card with RSP2, the bandwidth will drop down to 23 Gbps for each of the channels. Arbitration Another intelligent feature is that the line cards access to the RSP’s Switch fabric is controlled, each RSP has an arbiter ASIC acting as a super cop. The Arbiter on the Active RSP is Active and the other Arbiter silently listens. Before the FIA sends a packet to the switch fabric, it will send a fabric request to the Arbiter. The Arbiter will check if there is available bandwidth on the egress line card. If yes, it will grant access to the ingress FIA. Arbitration makes sure that there is bandwidth fairness among the Line cards. Load Balancing on Fabric Unicast and Multicast packets are load balanced in a different way. Unicast traffic is sent across the first available fabric link to the destination (maximizing efficiency). Each frame contains sequencing information and all destination FIAs have re-sequencing logic. Multicast traffic is hashed based on (S,G) info to maintain flow integrity. So for one (S,G) entry all the packets will take a particular fabric link. Troubleshooting Packet Loss on ASR 9000 One common problem we see across customer networks is packet loss in the system. In order to troubleshoot such issue on the ASR 9000, it’s very important to isolate where exactly the packets are getting lost. To start troubleshooting we should check the following for drops/errors, PLIM (Physical Layer Interface Module) NP (Network Processor) Bridge ASIC (For Generation 1 Line card) FIA (Fabric Interface ASIC) Switch Fabric Once the problem location is identified we can take the corrective action. Step 1:- Checking Traffic Loss at PLIM The first step to troubleshoot any Packet loss should be the Physical Interface which is PLIM. Following are the useful commands to check any kind of drops. 1.      show interface Gigabitethernet / Tengigabitethernet < > RP/0/RSP0/CPU0:CWG-ASR-4#show interfaces TenGigE0/0/0/1 Wed Apr 17 06:46:09.716 UTC TenGigE0/2/0/0 is administratively down, line protocol is administratively down Interface state transitions: 0 Hardware is TenGigE, address is 18ef.63e3.5b48 (bia 18ef.63e3.5b48) Layer 1 Transport Mode is LAN Internet address is Unknown MTU 1514 bytes, BW 10000000 Kbit (Max: 10000000 Kbit)      reliability 255/255, txload 0/255, rxload 0/255 Encapsulation ARPA, Full-duplex, 10000Mb/s, link type is force-up output flow control is off, input flow control is off loopback not set, ARP type ARPA, ARP timeout 04:00:00 Last input never, output never Last clearing of "show interface" counters never 5 minute input rate 0 bits/sec, 0 packets/sec 5 minute output rate 0 bits/sec, 0 packets/sec      0 packets input, 0 bytes, 0 total input drops      0 drops for unrecognized upper-level protocol      Received 0 broadcast packets, 0 multicast packets              0 runts, 0 giants, 0 throttles, 0 parity      0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort      0 packets output, 0 bytes, 0 total output drops      Output 0 broadcast packets, 0 multicast packets      0 output errors, 0 underruns, 0 applique, 0 resets      0 output buffer failures, 0 output buffers swapped out      0 carrier transitions From the above output look for the following. •a)      Input Drops •b)      Output drops •c)      CRC Errors •d)      Carrier Transitions •2.      RP/0/RSP0/CPU0:CWG-ASR-4#show controllers Tengigabitethernet < > stats show controllers tengigabitEthernet 0/0/0/1 stats | in drop Wed Apr 17 02:26:59.012 EDT Input drop overrun             = 0 Input drop abort               = 0 Input drop invalid VLAN     = 0 Input drop invalid DMAC     = 0 Input drop invalid encap   = 0 Input drop other                 = 0 Output drop underrun       = 0 Output drop abort             = 0 Output drop other              = 0 By checking the Drop counters you can verify if issue is with Ingress Direction or Egress Direction Step 2:- Checking Traffic Loss at Network Processor After Verifying the PLIM side the next step is to check the NP which is responsible for the Packet forwarding. All the Interface are connected to NPs and there are a number of NPs on each card. We should first check on which NP our Problematic Interface is connected. Following commands can be used. RP/0/RSP1/CPU0:ASR9010-A#sh controller np summ all Mon Apr 6 10:31:21.480 UTC                Node: 0/0/CPU0: ---------------------------------------------------------------- [total 4 NP] Driver - Version 10.26a Build 1010 ( Jan 28 2009, 12:00:00 ) NP 0 : Hardware Revision: v2 A1      : Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 ) NP 1 : Hardware Revision: v2 A1      : Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 ) NP 2 : Hardware Revision: v2 A1      : Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 ) NP 3 : Hardware Revision: v2 A1      : Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 )                Node: 0/5/CPU0: ---------------------------------------------------------------- [total 4 NP] Driver - Version 10.26a Build 1010 ( Jan 28 2009, 12:00:00 ) NP 0 : Hardware Revision: v2 A1      : Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 ) NP 1 : Hardware Revision: v2 A1      : Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 ) NP 2 : Hardware Revision: v2 A1      : Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 ) NP 3 : Hardware Revision: v2 A1      : Ucode - Version: 255.255 Build Date: ( Mar 19 2009, 21:56:00 ) So the above command shows that we have two line cards in the router, one in SLOT 0 and other in SLOT 5 and each card has 4 NP’s. The Next thing to check is which NP our interface belongs to. RP/0/RSP1/CPU0:ASR9010-A#show controller np ports all location 0/0/CPU0 Mon Apr 6 10:44:41.289 UTC                Node: 0/0/CPU0: ---------------------------------------------------------------- NP Bridge Fia                       Ports -- ------ --- --------------------------------------------------- 0 1     0   TenGigE0/0/0/3 1 1     0   TenGigE0/0/0/2 2 0     0   TenGigE0/0/0/1 3 0     0   TenGigE0/0/0/0 So From the above output we can see that Interface tengige0/0/0/1 belongs to NP3. Next we need to check the NP Counters. The ‘show controllers np…’ command displays information about counters that helps you troubleshoot drops in the LCs. The names of the internal NP counters have the general format STAGE_DIRECTION_ACTION, for example, PARSE_FABRIC_RECEIVE_CNT, RESOLVE_EGRESS_DROP_CNT, and MODIFY_FRAMES_PADDED_CNT. The values of stage, direction and action are as follows: •There are five stages in the NP: Parse Search-I Modify Search-II Resolve •Examples of the direction are: Ingress Egress Next_hop •Examples of the action are: Drop_count Down There are additional counters such as DROP, PUNT and DIAGS, that provide important information but are not associated with a specific internal NP stage. Drop and punt counters are kept as an aggregate total per stage. show controllers np counters np3 location 0/0/CPU0 | in DROP Mon Nov 15 12:18:35.289 EST    30 RESOLVE_INGRESS_DROP_CNT          9           0    31 RESOLVE_EGRESS_DROP_CNT             6           0 295 DROP_IPV4_NEXT_HOP_DOWN       15         0 Also, observe in the same command output packet sent and received counts, bytes transferred, packet counters categorized by packet size, and so forth. The fields of interest are: xaui_a_t_transmited_packets_cnt: The number of packets sent by the NP to the bridge xaui_a_r_received_packets_cnt: The number of packets sent by the bridge to the NP Step 3:- Checking Traffic Loss at Brig ASIC (Only for Generation 1 Line cards) Generation 1 line cards have the Bridge ASIC which Connects NP with FIA and following outputs should be captured for to see any issue. RP/0/RSP0/CPU0:router#show controllers fabric fia bridge stats location 0/0/CPU0 Wed Aug 25 14:12:03.916 DST Device   Rx Interface     Packet     Error       Threshold                                                          Count     Drops         Drops                     --------------------------------------------------------------------------- Bridge0 From-Fabric(DDR)   603698         0             0                              From CPU             711734         0             0 RP/0/RSP0/CPU0:router# show controllers fabric fia bridge stats location 0/0/CPU0 UC - Unicast ,         MC - Multicast LP - LowPriority ,     HP - HighPriority --------------------------------------------------------------------------------                                  FIA 0                                  ****** Cast/   Packet         Packet             Error     Threshold                    Prio   Direct         Count              Count       Count ------------------------------------------------------------------------------ Unicast Egress Stats ******************** UC HP   Fabric to NP-0   28                 0               0               UC LP   Fabric to NP-0   0                    0               0               UC HP   Fabric to NP-1   28                 0               0               UC LP   Fabric to NP-1   0                   0               0               UC HP   Fabric to NP-2   28                 0                0               UC LP   Fabric to NP-2   0                   0               0               UC HP   Fabric to NP-3   28                 0               0               UC LP   Fabric to NP-3   0                   0               0               ---------------------------------------------------------------- UC     Total Egress     112                 0               0               Multicast Egress Stats ********************* MC HP   Fabric to NP-0   205                 0               0               MC LP   Fabric to NP-0   2                   0               0               MC HP   Fabric to NP-1   205                 0               0               MC LP   Fabric to NP-1   2                   0               0               MC HP   Fabric to NP-2   205                 0               0               MC LP   Fabric to NP-2   2                   0               0               MC HP   Fabric to NP-3   205                 0               0               MC LP   Fabric to NP-3   2                   0               0               --------------------------------------------------------------- MC     Total Egress     828                 0               0               Step 4:-   Verifying Fabric Interface ASIC We have FIA on the Line card where can use the following commands to check the stats of FIA. show controllers fabric fia stats location 0/0/cpu0 | in drop                                Ingress drop:                       0                                  Egress drop:                     1020                                  Total drop:                     1020 Using this command we can check if issue is on ingress side or egress side. Similarly using the below mentioned commands we can check the errors on ingress and egress sides. RP/0/RSP0/CPU0:CWG-ASR-4#show controllers fabric fia errors ingress location 0/0/CPU0 ********** FIA-0 ********** Category: in_error-0                                  QDR Release                       0                                Latency FIFO-0                       0                                Latency FIFO-1                       0                                 DDR Rx Seq-0                       0                                  DDR Rx Seq-1                       0                                  DDR Rx Hdr-0                       0                                  DDR Rx Hdr-1                       0                                  DDR Rx CRC-0                       0                                  DDR Rx CRC-1                       0                                  DDR Rx Seq-0                       0                                  DDR Rx Seq-1                       0                                DDR Rx Lost-0                       0                                DDR Rx Lost-1                       0                                  DDR Rx Len-0                       0                                 DDR Rx Len-1                       0                                DDR Rx Hdr2-0                       0                                DDR Rx Hdr2-1                       0                                      Asm Hdr                             0                                  QDR Payload                       0                                QOS TailDrop-0                       0                                QOS TailDrop-1                       0                               QOS TailDrop-2                       0                                QOS TailDrop-3                       0                                        Uc Req                             0                                      Uc Grant                            0 RP/0/RSP0/CPU0:CWG-ASR-4# RP/0/RSP0/CPU0:CWG-ASR-4#show controllers fabric fia errors egress location 0/0/CPU0 ********** FIA-0 ********** Category: eg_error-0                                DDR Tx Prot-0                        0                                DDR Tx Prot-1                       0                                  DDR Tx CRC-0                       0                                  DDR Tx CRC-1                       0                                  DDR Tx Hdr-0                       0                                  DDR Tx Hdr-1                       0                                Egr RxFab Uc-0                       0                                Egr RxFab Uc-1                       0                               Egr RxFab Uc-2                       0                                Egr RxFab Uc-3                       0                                Egr RxFab Mc-0                       0                                Egr RxFab Mc-1                      0                                Egr RxFab Mc-2                       0                                Egr RxFab Mc-3                       0                                RxFab Uc Len-0                       0                                RxFab Uc Len-1                       0                                RxFab Uc Len-2                       0                                RxFab Uc Len-3                       0                                RxFab Mc Len-0                       0                                RxFab Mc Len-1                       0                                RxFab Mc Len-2                       0                                RxFab Mc Len-3                       0                               RxFab Pkt McL0                       0                                RxFab Pkt McL1                       0                                RxFab Pkt McL2                       0                                RxFab Pkt McL3                        0                                RxFab Uc CRC-0                       0                                RxFab Uc CRC-1                       0                                RxFab Uc CRC-2                       0                               RxFab Uc CRC-3                       0                                RxFab Mc CRC-0                       0                                RxFab Mc CRC-1                       0                                RxFab Mc CRC-2                       0                               RxFab Mc CRC-3                       0                                RxFab2 UcLen-0                       0                                RxFab2 UcLen-1                       0                                RxFab2 UcLen-2                       0                                RxFab2 UcLen-3                       0                                RxFab2 McLen-0                       0                                RxFab2 McLen-1                       0                               RxFab2 McLen-2                       0                                RxFab2 McLen-3                       0                                      Mc Deq-0                       0                                      Mc Deq-1                        0 RP/0/RSP0/CPU0:CWG-ASR-4# Step 5:-   Verifying Switch Fabric Verify the Switch fabric for any errors and packet drops. RP/0/RSP0/CPU0:CWG-ASR-4# RP/0/RSP0/CPU0:CWG-ASR-4#show controllers fabric crossbar statistics instance 0 location 0/CPU0/0 Port statistics for xbar:0 port:0 ============================== Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:1 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:2 ============================== Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:3 ============================== Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:4 ============================== Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:5 ============================== Internal Error Count: 3 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:6 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:7 ============================== Internal Error Count: 1 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:8 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:9 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:10 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:11 ============================== Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:12 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:13 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:15 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:16 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:17 ============================== Internal Error Count: 4 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Port statistics for xbar:0 port:18 ============================== Internal Error Count: 1 Hi priority stats (unicast) =========================== Low priority stats (multicast) =========================== Total Unicast In:     0 Total Unicast Out:   0 Total Multicast In:  0 Total Multicast Out: 0 I hope this has been an informative session and proves useful for you. Please do share your feedback and opinion via the comments session below. Thank you for reading & watching! ... View more

Community Tech-Talk series is designed to bring technical Experts from Cisco to  share their insights on specific topics, selectively chosen based on  most-common conversation themes on our technology area on community from  our technology communities. This  Tech-Talk focuses on Understanding Life of a Packet on ASR9k and also understanding how to troubleshoot packet loss issue on ASR9k. Session Objectives: Basic Architecture Overview Understanding Unicast and Multicast packet flow Troubleshooting Packet loss Issues, Watch the Tech-Talk and checkout the related Blog I hope this has been an informative session and proves useful for understanding Life of a Packet and Troubleshooting Packet Loss on ASR9k. Please do share your feedback and opinion via  the comments session below. For more insight to Cisco ASR9k Architecture and best practices on troubleshooting please visit the below URLs ASR9K Architecture Troubleshooting Fabric on ASR9k ... View more