Introduction OSPF, being a link-state protocol, allows for every router in the network to know of every link and OSPF speaker in the entire network. From this picture each router independently runs the Shortest Path First (SPF) algorithm to determine the best path through the network. All of this information is stored in the "Link State Database" (LSDB). Every network engineer has seen the LSDB at some point by running show ip ospf database but few actually know how to read the details. By looking only at the LSDB we should have enough information to draw a topology diagram from scratch. Link State Advertisements OSPF uses "Link State Advertisements" (LSAs) to provide information about links and link-costs to neighboring OSPF speakers. OSPF defines multiple LSAs, which all serve a different purpose. LSA Type Who Generates the LSA? What is Accomplished? Type 1 - Router LSA Every router in every area How routers advertise their connected interfaces Type 2 - Network LSA DRs on all non-point-to-point links The DR collects all the Type 1 LSAs and sends out a single Type 2 representing all of the routers on the link. This is used to build the Shortest Path Tree Type 3 - Network Summary LSA Area Border Routers (ABRs) ABRs send a single LSA representing all of the Type 1 and Type 2 LSAs in an area. This reduces the number of LSAs on the routers in other areas. Type 4 - ASBR Summary LSA ABRs connected to an area where external routes (Type 5) are originated Type 4 LSAs are sent to other areas to build the Shortest Path Tree to an ASBR. Type 5 - AS External LSA Routers with the redistribute command that are not in a NSSA area This represents any external routes redistributed into OSPF. Type 7 - NSSA External LSA Routers with the redistribute command that are in a NSSA area Type 5 LSAs are not allowed in Stub Areas. Type 7 LSAs allow external information to pass through NSSA areas. Building the Topology Starting on a router named r120 we can get a high level overview of the network (or at least our Area). First, who are we (what is our Router ID)? r120#show ip ospf data             OSPF Router with ID ( 10.0.0.120 ) (Process ID 1) Next, who are the other routers in our area?                 Router Link States (Area 1) Link ID         ADV Router      Age         Seq#       Checksum Link count 10.0.0.111       10.0.0.111      600         0x8000023A 0x0092B3 1 10.0.0.112       10.0.0.112      1246        0x80000234 0x009CAC 1 10.0.0.113       10.0.0.113      148         0x8000022C 0x004399 3 10.0.0.120       10.0.0.120      152         0x80000240 0x0046CB 1 This tells us there are four routers in Area 1. The router with RID 10.0.0.113 has 3 links in Area 1, every one else has only 1 link. Next, who are all of the DRs in this Area? What network segments do they represent?                 Net Link States (Area 1) Link ID         ADV Router      Age         Seq#       Checksum 192.168.1.112    10.0.0.112       1862        0x80000237 0x00D860 192.168.7.113    10.0.0.113       12          0x80000001 0x00E8F5 Routers 10.0.0.112 and 10.0.0.113 are the router IDs of the DRs for two segments . 192.168.1.112 and 192.168.7.113 represent the IP address of the DR on that segment. We will see later that if we were to look at the Router LSA for 10.0.0.112, for example, we would see 192.168.1.112 as one of the interfaces owned by that router. The Summary Network LSAs (Type 3) are generated by the ABRs and will give us information about every segment in the network, outside of our Area. Type 1 and Type 2 LSAs are not flooded beyond an ABR. The ABR is responsible for taking all of the information in Type 1 and Type 2 LSAs and repackaging them into Type 3 LSAs.                 Summary Net Link States (Area 1) Link ID         ADV Router      Age         Seq#       Checksum 10.0.0.119      10.0.0.111       1215        0x8000022A 0x00A845 10.0.0.119      10.0.0.112       1862        0x80000229 0x00A449 192.168.0.0     10.0.0.111       1215        0x80000234 0x00D842 192.168.0.0     10.0.0.112       1862        0x80000233 0x00D446 192.168.2.0     10.0.0.111      1215        0x80000234 0x0027E7 192.168.2.0     10.0.0.112       1862        0x80000233 0x0023EB 192.168.3.0     10.0.0.111       1215        0x80000232 0x008481 192.168.3.0     10.0.0.112       1862        0x80000232 0x007E86 192.168.4.0     10.0.0.111      1215        0x80000232 0x00798B 192.168.4.0     10.0.0.112       1862        0x80000232 0x007390 192.168.5.0     10.0.0.111       1215        0x80000232 0x006E95 192.168.5.0     10.0.0.112       1862        0x80000232 0x00689A 192.168.6.0     10.0.0.111       1215        0x80000231 0x00C930 192.168.6.0     10.0.0.112       1862        0x80000231 0x00C335 From this we know Area 1 has two ABRs with RIDs 10.0.0.111 and 10.0.0.112 . We also see a total of 7 segments in the entire OSPF network. Each network is seen twice because each ABR generates its own LSA. Two ABRs means two LSAs. Type 4, Summary Network LSAs are next. These are generated by the ABRs ( 10.0.0.111 and 10.0.0.112 ) to represent any routers or ABRs outside of our Area that is passing along Type 5 (external LSAs). This may be a little confusing at this point but it will make more sense when we start working through the topology. Summary ASB Link States (Area 1) Link ID         ADV Router      Age         Seq#       Checksum 10.0.0.114      10.0.0.111      1215        0x80000232 0x00E915 10.0.0.114      10.0.0.112      1862        0x80000232 0x00E31A Finally the external routes are represented by Type 5 LSAs.                 Type-5 AS External Link States Link ID         ADV Router      Age         Seq#       Checksum Tag 172.16.0.113    10.0.0.113      631         0x80000001 0x00F006 0 172.16.0.118    10.0.0.114      678         0x80000001 0x009775 0 Here we see two different prefixes being redistributed into OSPF. The ADV Router is either the configured with the redistribute command (10.0.0.113) or an ABR connected to a NSSA area where redistribution is happening. With this information let's build some high level topologies. First, start with what we know from the Type 1 LSAs (all the routers in our area) We will skip the Type 2 Network LSAs for now and go to the Type 3 Summary Network LSAs. This gives us the other subnets in the network and the ABRs for Area 1 The Type 4 ASBR Summary LSAs let us know that 10.0.0.114 is also an ABR on Area 0 and finally, the Type 5's tell us about the externals. Using the "ADV Router" field we can figure out where those routes come from. Just from looking at the summary information in the database we've been able to put a lot of information together. Now we can start looking into the LSAs to get an idea of what the connectivity of Area 1 looks like. We will start with router r120. Since we don't know anything about the links r120 has, we start with a router with no connections. To see the connections on r120 we'll need to look at the Router LSA that is generated by r120 (remember: Router LSAs are represented by the Router ID) r120#show ip ospf database router 10.0.0.120             OSPF Router with ID ( 10.0.0.120 ) (Process ID 1)                 Router Link States (Area 1)   LS age: 408   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.120   Advertising Router: 10.0.0.120   LS Seq Number: 8000023C   Checksum: 0x815   Length: 36   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.7.113      (Link Data) Router Interface address: 192.168.7.120       Number of MTID metrics: 0        TOS 0 Metrics: 10 From this we know a few things: 1.) There is a single interface with IP 192.168.7.120 2.) There is a least one other router on this segment with IP 192.168.7.113 , and this is the DR 2a.) Because there is a DR, we know this interface is multi-access (not point-to-point) 3.) The Router ID ( 10.0.0.120 ) is not advertised in OSPF (becuase there is no link information representing the router ID) 4.) The metric we are advertising is 10 We have an IP and a DR, but we don't know the subnet mask or which router in Area 1 owns the DR IP address. There is where the Type 2 LSA comes in. Remember the Type 2 is generated by the DR for a segment, and represents that segment, so we look for the segment DR. r120#show ip ospf data network 192.168.7.113             OSPF Router with ID (10.0.0.120) (Process ID 1)                 Net Link States (Area 1)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 93   Options: (No TOS-capability, DC)   LS Type: Network Links   Link State ID: 192.168.7.113 (address of Designated Router)   Advertising Router: 10.0.0.113   LS Seq Number: 80000004   Checksum: 0xE2F8   Length: 32   Network Mask: /24         Attached Router : 10.0.0.113         Attached Router : 10.0.0.120 A lot of great information here. First, we see the advertising router, which is the Router ID of the DR. In this case it's 10.0.0.113 . Now we know who r120 is attached to. We also see the network mask (/24) and all of the routers on the segment. In this case only r120 and 10.0.0.113 are on the segment. If there were other routers on this segment we would see their Router IDs in the " Attached Router " list. So let's update the topology diagram. Now we can look at the Router LSA of 10.0.0.113 r120#show ip ospf data router 10.0.0.113             OSPF Router with ID (10.0.0.120) (Process ID 1)                 Router Link States (Area 1)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 395   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.113   Advertising Router: 10.0.0.113   LS Seq Number: 80000256   Checksum: 0x5465   Length: 60   AS Boundary Router   Number of Links: 3     Link connected to: a Stub Network      (Link ID) Network/subnet number: 10.0.0.113      (Link Data) Network Mask: 255.255.255.255       Number of MTID metrics: 0        TOS 0 Metrics: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.7.113      (Link Data) Router Interface address: 192.168.7.113       Number of MTID metrics: 0        TOS 0 Metrics: 10     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.1.112      (Link Data) Router Interface address: 192.168.1.113       Number of MTID metrics: 0        TOS 0 Metrics: 10 Here we see three interfaces: 10.0.0.113 , 192.168.7.113 and 192.168.1.113 . We see that 10.0.0.113 is a "Stub Network". This does not have any relationship to a Stub Area, a "stub network" is simply an interface with no OSPF neighbors on it. We can also see that we are not the DR on the segment for 192.168.1.113 . Let's take a look at the Type 2 for that segment. Remember, the Type 2 is represented by the DR for that segment. r120#show ip ospf data network 192.168.1.112             OSPF Router with ID (10.0.0.120) (Process ID 1)                 Net Link States (Area 1)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 161   Options: (No TOS-capability, DC)   LS Type: Network Links   Link State ID: 192.168.1.112 (address of Designated Router)   Advertising Router: 10.0.0.112   LS Seq Number: 80000261   Checksum: 0x848A   Length: 36   Network Mask: /24         Attached Router: 10.0.0.112         Attached Router: 10.0.0.111         Attached Router: 10.0.0.113 We know that routers 10.0.0.112 , 10.0.0.111 and 10.0.0.113 are all attached to this segment. Now we can look at the Router LSAs for routers 10.0.0.112 and 10.0.0.111 . This will provide us with their interface IPs as well as any Stub Networks we haven't seen yet. r120#show ip ospf data router 10.0.0.111             OSPF Router with ID (10.0.0.120) (Process ID 1)                 Router Link States (Area 1)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 1004   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.111   Advertising Router: 10.0.0.111   LS Seq Number: 80000264   Checksum: 0x3EDD   Length: 36   Area Border Router   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.1.112      (Link Data) Router Interface address: 192.168.1.111       Number of MTID metrics: 0        TOS 0 Metrics: 10 r120#show ip ospf data router 10.0.0.112             OSPF Router with ID (10.0.0.120) (Process ID 1)                 Router Link States (Area 1)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 1444   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.112   Advertising Router: 10.0.0.112   LS Seq Number: 8000025E   Checksum: 0x48D6   Length: 36   Area Border Router   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.1.112      (Link Data) Router Interface address: 192.168.1.112       Number of MTID metrics: 0        TOS 0 Metrics: 10 Now we know that the DR, 192.168.1.112 , is router 10.0.0.112 . We know that router 10.0.0.111 is also connected to the 192.168.1.0/24 segment with IP 192.168.1.111 . We now know the entire topology for Area 1. We know everything there is to know about Area 1. There is nothing to learn from router 10.0.0.113, since all of the links on that router are discovered. The next point to continue mapping the network would be on one of the ABRs. We will start with 10.0.0.112, or r112. Since r112 is an ABR it will have Type 1, 2, 3 and 4 information for both Area 1 and Area 0. We will want to focus on the Area 0 information. Let's get started by looking at our own Router LSA        OSPF Router with ID (10.0.0.112) (Process ID 1)                 Router Link States (Area 0)   LS age: 720   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.112   Advertising Router: 10.0.0.112   LS Seq Number: 80000262   Checksum: 0x20FD   Length: 36   Area Border Router   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.0.111      (Link Data) Router Interface address: 192.168.0.112       Number of MTID metrics: 0        TOS 0 Metrics: 10 Looks like we only have link 192.168.0.112 in this area, and we are not the DR on this segment. Now we take a look at the Type 2 LSA for this segment. r112#show ip ospf data network 192.168.0.111             OSPF Router with ID (10.0.0.112) (Process ID 1)                 Net Link States (Area 0)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 388   Options: (No TOS-capability, DC)   LS Type: Network Links   Link State ID: 192.168.0.111 (address of Designated Router)   Advertising Router: 10.0.0.111   LS Seq Number: 80000261   Checksum: 0x759F   Length: 36   Network Mask: /24         Attached Router: 10.0.0.111         Attached Router: 10.0.0.110         Attached Router: 10.0.0.112 We see that the advertising router represents the Router ID of the DR, or 10.0.0.111 , who we already know is the other ABR for Area 1. We also see that there is a third router on this segment with Router ID 10.0.0.110 .  Let's get the interface information from 10.0.0.111 r112#show ip ospf data router 10.0.0.111             OSPF Router with ID (10.0.0.112) (Process ID 1)                 Router Link States (Area 0)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 700   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.111   Advertising Router: 10.0.0.111   LS Seq Number: 80000268   Checksum: 0x1605   Length: 36   Area Border Router   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.0.111      (Link Data) Router Interface address: 192.168.0.111       Number of MTID metrics: 0        TOS 0 Metrics: 10 One interface here, with IP 192.168.0.111 . This is the same segment as r112 and Router 10.0.0.110. Finally, let's look at 10.0.0.110 r112#show ip ospf data router 10.0.0.110             OSPF Router with ID (10.0.0.112) (Process ID 1)                 Router Link States (Area 0)   LS age: 1232   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.110   Advertising Router: 10.0.0.110   LS Seq Number: 80000263   Checksum: 0x4E09   Length: 48   Number of Links: 2     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.0.111      (Link Data) Router Interface address: 192.168.0.110       Number of MTID metrics: 0        TOS 0 Metrics: 10     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.2.110      (Link Data) Router Interface address: 192.168.2.110       Number of MTID metrics: 0        TOS 0 Metrics: 10 We see that 10.0.0.110 owns the IP 192.168.0.110 . We also see a second interface, 192.168.2.110 . On this second segment 10.0.0.110 is the DR. Let's up the topology diagram for Area 0: Let's keep moving down. First, we look at the Type 2 from 192.168.2.110 , then we'll look at the Type 1 LSAs from the other routers on this segment. r112#show ip ospf data net 192.168.2.110             OSPF Router with ID (10.0.0.112) (Process ID 1)                 Net Link States (Area 0)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 781   Options: (No TOS-capability, DC)   LS Type: Network Links   Link State ID: 192.168.2.110 (address of Designated Router)   Advertising Router: 10.0.0.110   LS Seq Number: 80000261   Checksum: 0x1779   Length: 32   Network Mask: /24         Attached Router: 10.0.0.110         Attached Router: 10.0.0.114 And now the Type 1 for 10.0.0.114 r112#show ip ospf data router 10.0.0.114             OSPF Router with ID (10.0.0.112) (Process ID 1)                 Router Link States (Area 0)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 889   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.114   Advertising Router: 10.0.0.114   LS Seq Number: 80000265   Checksum: 0x1178   Length: 48   Area Border Router   AS Boundary Router   Number of Links: 2     Link connected to: a Stub Network      (Link ID) Network/subnet number: 192.168.3.0      (Link Data) Network Mask: 255.255.255.0       Number of MTID metrics: 0        TOS 0 Metrics: 10     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.2.110      (Link Data) Router Interface address: 192.168.2.114       Number of MTID metrics: 0        TOS 0 Metrics: 10 We have two links again, one connected to the segment with 10.0.0.110 and a new segment. Again, notice that the segment 192.168.3.0 is a Stub Network, so there are no other OSPF speakers on this link. Now, before we think we've finished up, we haven't looked at the Type 3 LSAs that are generated by ABRs. We don't know if there is another ABR in Area 0, so let's look                 Summary Net Link States (Area 0) Link ID         ADV Router      Age         Seq#       Checksum 10.0.0.113      10.0.0.111      397         0x80000264 0x004394 10.0.0.113      10.0.0.112      17          0x80000264 0x003D99 10.0.0.119      10.0.0.114       1276        0x80000258 0x00715F 192.168.1.0     10.0.0.111      397         0x80000263 0x006F7B 192.168.1.0     10.0.0.112      792         0x80000262 0x006B7F 192.168.4.0     10.0.0.114       1036        0x80000261 0x0040A6 192.168.5.0     10.0.0.114       1036        0x80000261 0x0035B0 192.168.6.0     10.0.0.114       1036        0x80000260 0x00904B 192.168.7.0     10.0.0.111      397         0x80000265 0x008D4B 192.168.7.0     10.0.0.112      17          0x80000265 0x008750 Before addressing the new routes here, you can see the Type 3 LSAs in Area 0 that are generated by the two ABRs, 10.0.0.111 and 10.0.0.112. These routes here are all of the routes in Area 1, that we just described. This is how an ABR hides the details of an Area from the rest of the network. We see four new networks all coming from the ABR 10.0.0.114. Now we can update our topology diagram of Area 0. Again, we need to jump to our ABR to see what's going on in the rest of the network. On r114 things get interesting. Looking at the LSAs we see that r114 is in 3 areas. r114# show ip ospf data | i States                 Router Link States (Area 0)                 Net Link States (Area 0)                 Summary Net Link States (Area 0)                 Summary ASB Link States (Area 0)                 Router Link States (Area 2)                 Net Link States (Area 2)                 Summary Net Link States (Area 2)                 Router Link States (Area 3)                 Net Link States (Area 3)                 Summary Net Link States (Area 3)                 Type-7 AS External Link States (Area 3)                 Type-5 AS External Link States But things look a little fishy in Area 2 and Area 3. Notice that Area 2 has no "Summary ASB Link States" (Type 4). Also notice that Area 3 has "Type-7 AS External Link States". Let's start with Area 2. If we are in an area that does not have any Type 4 LSAs, that area can not have external routes. OSPF works by linking the information carried in a Type-4 LSA to the information carried in the Type-5 LSA to build a tree. OSPF Stub areas do not allow any external information, matching this description. We can assume that Area 2 is a Stub Area. Again, let's get a lay of the land by looking at the Router LSA summaries.                 Router Link States (Area 2) Link ID         ADV Router      Age         Seq#       Checksum Link count 10.0.0.114      10.0.0.114      23          0x8000026C 0x00B250 1 10.0.0.115      10.0.0.115      1584        0x80000269 0x00B350 1 10.0.0.116      10.0.0.116      745         0x8000026E 0x00F225 2 10.0.0.119      10.0.0.119      706         0x8000026B 0x0074E7 2 We can see there are four routers in the area. 10.0.0.116 and 10.0.0.119 both have two links. Now let's look at our Router LSA r114#show ip ospf data router 10.0.0.114                      Router Link States (Area 2)   LS age: 1234   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.114   Advertising Router: 10.0.0.114   LS Seq Number: 80000265   Checksum: 0xC049   Length: 36   Area Border Router   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.5.114      (Link Data) Router Interface address: 192.168.5.114       Number of MTID metrics: 0        TOS 0 Metrics: 10 Now the Type-2, Network LSA r114#show ip ospf data net 192.168.5.114             OSPF Router with ID (10.0.0.114) (Process ID 1)                 Net Link States (Area 2)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 1312   Options: (No TOS-capability, DC)   LS Type: Network Links   Link State ID: 192.168.5.114 (address of Designated Router)   Advertising Router: 10.0.0.114   LS Seq Number: 80000263   Checksum: 0xD429   Length: 36   Network Mask: /24         Attached Router: 10.0.0.114         Attached Router: 10.0.0.115         Attached Router: 10.0.0.116 Next, the Router LSA (Type-1) of our Attached Routers r114#show ip ospf data router 10.0.0.115             OSPF Router with ID (10.0.0.114) (Process ID 1)                 Router Link States (Area 2)   LS age: 1000   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.115   Advertising Router: 10.0.0.115   LS Seq Number: 80000263   Checksum: 0xBF4A   Length: 36   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.5.114      (Link Data) Router Interface address: 192.168.5.115       Number of MTID metrics: 0        TOS 0 Metrics: 10 r114#show ip ospf data router 10.0.0.116             OSPF Router with ID (10.0.0.114) (Process ID 1)                 Router Link States (Area 2)   LS age: 269   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.116   Advertising Router: 10.0.0.116   LS Seq Number: 80000268   Checksum: 0xFE1F   Length: 48   Number of Links: 2     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.5.114      (Link Data) Router Interface address: 192.168.5.116       Number of MTID metrics: 0       TOS 0 Metrics: 10     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.6.119      (Link Data) Router Interface address: 192.168.6.116       Number of MTID metrics: 0        TOS 0 Metrics: 10 Let's digest this here. First 10.0.0.115 has a single interface with IP 192.168.5.115 . Next we see 10.0.0.116 with two interfaces, 192.168.5.116 and 192.168.6.116 . Let's update the topology and then take a look at the segment with DR 192.168.6.119. This must be the router 10.0.0.119, the only router in Area 2 we haven't looked at yet. Remember that routers 10.0.0.116 and 10.0.0.119 both had two links. We have discovered the two links on 10.0.0.116, but we still have one link on 10.0.0.119 to find, so let's look at the Type-1 for 10.0.0.119 r114#show ip ospf data router 10.0.0.119             OSPF Router with ID (10.0.0.114) (Process ID 1)                 Router Link States (Area 2)   LS age: 1272   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.119   Advertising Router: 10.0.0.119   LS Seq Number: 8000026B   Checksum: 0x74E7   Length: 48   Number of Links: 2     Link connected to: a Stub Network      (Link ID) Network/subnet number: 10.0.0.119      (Link Data) Network Mask: 255.255.255.255       Number of MTID metrics: 0        TOS 0 Metrics: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.6.119      (Link Data) Router Interface address: 192.168.6.119       Number of MTID metrics: 0        TOS 0 Metrics: 10 Now we have discovered all of the links in Area 2. Because Area 2 still receives Type-3 LSAs, it will know about all of the links in the OSPF network. The only thing it will not have will be the external routes that are injected into OSPF. Remember that r114 was in three Areas: Area 0, Areas 2 and Area 3. Before digging into Area 3, let's take another look at the LSAs that exist in Area 3. r114#show ip ospf data | i Area 3                 Router Link States (Area 3)                 Net Link States (Area 3)                 Summary Net Link States (Area 3)                 Type-7 AS External Link States (Area 3) We see Type 1 ( Router Link States ), Type 2 ( Net Link State ), Type 3 ( Summary Net Link ) and Type 7 ( Type-7 AS External ). Similar to Area 2, we do not see Type 4 or Type 5 LSAs. However we see Type-7 LSAs, which only exist in Not So Stubby Areas (NSSA). In a normal Stub area external route information is not allowed. NSSA areas allow us to have all of the features of a Stub area (no externals from other parts of the network) while still allowing external information to be originated in this area. To accomplish this, NSSA areas do not allow Type-5 (normal external LSAs) and use a special Type-7 LSAs. When the Type-7 arrives on the ABR (r114 in this case), the ABR must convert this Type-7 to a Type-5 for the rest of the network. We'll take a look at this process in a little while. First, let's see how many routers and links are in Area 3                Router Link States (Area 3) Link ID         ADV Router      Age         Seq#       Checksum Link count 10.0.0.114      10.0.0.114      1610        0x80000335 0x00B37A 1 10.0.0.117      10.0.0.117      1344        0x80000333 0x00A881 1 10.0.0.118      10.0.0.118      802         0x80000332 0x00AE77 1 We have 3 routers, each with 1 link. Now, as always, take a look at our Type-1 r114#show ip ospf data router 10.0.0.114                 Router Link States (Area 3)   LS age: 723   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.114   Advertising Router: 10.0.0.114   LS Seq Number: 80000334   Checksum: 0xB579   Length: 36   Area Border Router   AS Boundary Router   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.4.117      (Link Data) Router Interface address: 192.168.4.114       Number of MTID metrics: 0        TOS 0 Metrics: 10 A single link with IP 192.168.4 . 114 and DR 192.168.4.117 . Now the Type-2 r114#show ip ospf data net 192.168.4.117             OSPF Router with ID (10.0.0.114) (Process ID 1)                 Net Link States (Area 3)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 635   Options: (No TOS-capability, DC)   LS Type: Network Links   Link State ID: 192.168.4.117 (address of Designated Router)   Advertising Router: 10.0.0.117   LS Seq Number: 80000330   Checksum: 0xCE50   Length: 36   Network Mask: /24         Attached Router: 10.0.0.117         Attached Router: 10.0.0.114         Attached Router: 10.0.0.118 Here are three routers attached to this segment. r114, the DR and a third router. Now the Type-1 LSAs for the other routers. r114#show ip ospf data router 10.0.0.117             OSPF Router with ID (10.0.0.114) (Process ID 1)                 Router Link States (Area 3)   LS age: 794   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.117   Advertising Router: 10.0.0.117   LS Seq Number: 80000333   Checksum: 0xA881   Length: 36   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.4.117      (Link Data) Router Interface address: 192.168.4.117       Number of MTID metrics: 0        TOS 0 Metrics: 10 r114#show ip ospf data router 10.0.0.118             OSPF Router with ID (10.0.0.114) (Process ID 1)                 Router Link States (Area 3)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 257   Options: (No TOS-capability, DC)   LS Type: Router Links   Link State ID: 10.0.0.118   Advertising Router: 10.0.0.118   LS Seq Number: 80000332   Checksum: 0xAE77   Length: 36   AS Boundary Router   Number of Links: 1     Link connected to: a Transit Network      (Link ID) Designated Router address: 192.168.4.117      (Link Data) Router Interface address: 192.168.4.118       Number of MTID metrics: 0        TOS 0 Metrics: 10 We see IPs 192.168.4.118 and 192.168.4.117 . Here's the topology for Area 3. But let's not forget about the Type-7 LSAs we saw earlier. Because these are Type-7, we are not looking at external LSAs but nssa-external LSAs r114#show ip ospf data nssa-external                      OSPF Router with ID (10.0.0.114) (Process ID 1)                 Type-7 AS External Link States (Area 3)   Routing Bit Set on this LSA in topology Base with MTID 0   LS age: 952   Options: (No TOS-capability, Type 7/5 translation, DC)   LS Type: AS External Link   Link State ID: 172.16.0.118 (External Network Number )   Advertising Router: 10.0.0.118   LS Seq Number: 800000FF   Checksum: 0xEC13   Length: 36   Network Mask: /32         Metric Type: 2 (Larger than any link state path)         MTID: 0         Metric: 20         Forward Address: 192.168.4.118         External Route Tag: 0 This is the LSA representing the external network 172.16.0.118/32 . Within this area the routers should send traffic for this destination to the Forwarding Address of 192.168.4.118 . Once the LSA arrives on the ABR, r114, it will be converted into a Type-5 LSA and sent to all other areas (that aren't stubs, meaning that Area 2 will not see this LSA). We can confirm this by looking at the Type-5 LSAs r114#show ip ospf data external 172.16.0.118             OSPF Router with ID (10.0.0.114) (Process ID 1)                 Type-5 AS External Link States   LS age: 146   Options: (No TOS-capability, DC)   LS Type: AS External Link   Link State ID: 172.16.0.118 (External Network Number )   Advertising Router: 10.0.0.114   LS Seq Number: 80000101   Checksum: 0x9477   Length: 36   Network Mask: /32         Metric Type: 2 (Larger than any link state path)         MTID: 0         Metric: 20         Forward Address: 192.168.4.118         External Route Tag: 0 Here we see the Type-5 originated not by 10.0.0.118 , like the Type-7, but by 10.0.0.114 . This is due to the Type-7 to Type-5 conversion. Since r114 is generating a new LSA it sets itself as the Advertising Router. You'll also notice that the Forwarding Address has remained the same. When other routers in the network build the tree to reach this external destination they will build to the best ABR to reach this network (since it would be part of a Type-3 LSA). For more information on Forwarding Addresses, there is a great doc on Cisco.com. With all of this information we can finish the topology for Area 3 and then for the entire OSPF network. Summary Hopefully reading the OSPF topology is a little more clear now. The less obvious takeaways are how OSPF scales by hiding topology information. You noticed that in an area we have a large number of Type 1 and Type 2 LSAs. Outside of that area there is only a single Type-3 LSA generated by each ABR. We can also use Stub Areas to hide external information, keeping even less information in the LSDB of the routers in those areas. Finally, think about how each LSA type links together. OSPF's SPF algorithm links different pieces of information together. For a router in Area 1 to reach the external route in Area 3, it has to look at the Type-5 that represents the external route. Then it has to look at the Type-4 representing the ABR on the area that the ASBR lives in. Then we have to look at the Type-3 to get to that remote ABR. Finally we look at the Type-1 and Type-2 LSAs in our area to determine how to get to our closest ABR. Each LSA serves a specific purpose and they all fit together to supply end-to-end connectivity. ... View more

We all know that we our providers require BGP to connect to the internet, but it's not always so clear on how to fit this into the rest of our enterprise. Things get particularly tricky when we want to connect to the internet in different geographic regions and still provide optimal routing in and out of the enterprise. Background To start, let's focus on a network with both a North American and Asian presence. The company would like to have internet access both Asian and North American data centers, and allow regional users to use the local internet connection. Challenge 1: Number of Routes When setting up a new peering agreement with an ISP we need to decide how many routes we want to receive. There are generally three options: Full BGP Routes: The first option is to receive what is common called a "full BGP feed" or the entire "default free zone table". As of writing this, in early February 2013, there are roughly 443,000 prefixes on the internet. This number will have some slight differences between providers, but this is a pretty good ballpart number. The downside of receiving a full BGP feed from a provider is that it requires very large forwarding tables and very large amounts of memory. You will also feel all of the instability of the internet which will cause increased CPU usage to recalcuate and update the BGP table. The upside is that when you are connected to multiple providers you know the shortest path (number of AS hops) to any possible destination on the internet. Customer BGP Routes: A step down from a full BGP table is to only receive customer routes. In this case, you only receive the routes of other networks directly peered to the same ISP (and generally a default route as well). The number of routes received will vary widly from ISP to ISP but generally you will only be looking at a few thousand to around ten thousand routes. This can still be significant, but is an order of magnitude less than the full table. The big upside is that this takes a lot less memory and CPU to maintain and can provide more optimal routing than simple default routes. The downside is that it is possible that a majority of your internet bound traffic isn't going to a direct peer of your ISP, which would cause you rely on the default route anyhow. Default Only: This is the simplest solution. Have your provider send you a BGP default route and nothing more. You send all internet bound traffic to the ISP(s) and let them figure out the rest. This allows you to have routers with much smaller memory and CPU footprints at your internet edge. So how do you decide what is best for you? Like everything, it depends on your business requirements, but the simplest question to ask is, is a few miliseconds of latency to any given internet site acceptable? Another way to ask this is, do you mind if your internet routing is a little suboptimal? For most internet connectivity that is not providing cloud or web based tools this isn't a problem ... View more

One of the biggest challenges in troubleshooting is understanding expected behavior. If we don't know what it is supposed to look like in the working state, how can we tell if we are in a broken state? In this document we will explore label operations hop-by-hop through three common topologies for Traffic Engineering tunnels. These examples carry customer pseudowires (l2vpn) but the same concepts would apply for L3VPN as well. PE to PE Tunnel For our first example we have a pseudowire between PE2 and PE4. There is also a pair of TE tunnels that run end-to-end between PE2 and PE4. Let's look at the label operations from PE2 to PE4 First, we verify the labels on the pseudowire PE2 #show mpls l2 vc detail Local interface: Et0/1 up, line protocol up, Ethernet up   Destination address: 14.14.14.14, VC ID: 24, VC status: up     Output interface: Tu0, imposed label stack { 41 31 }     Preferred path: Tunnel0,  active     Default path: ready     Next hop: point2point We can see that two labels get imposed: The LDP label, 41 and the Pseudowire Label, 31 Since we are using a TE Tunnel (as you can see above, " Output interface: Tu0 ") the LDP label will be called the  "TE Label" or "Tunnel Label". We can dive a little deeper into the tunnel and see the Tunnel Instance , Outgoing Tunnel Label and exact path PE2 #show mpls traffic-eng tunnels tunnel 0 Name: PE2_t0                              (Tunnel0) Destination: 14.14.14.14   Status:     Admin: up         Oper: up     Path: valid       Signalling: connected     path option 1, type dynamic (Basis for Setup, path weight 50)   InLabel  :  -   OutLabel : Ethernet0/0, 41   Next Hop : 2.0.0.2   RSVP Signalling Info:        Src 22.22.22.22, Dst 14.14.14.14, Tun_Id 0, Tun_Instance 203     RSVP Path Info:       My Address: 2.0.0.12         Explicit Route: 2.0.0.2 4.0.0.2 4.0.0.4 6.0.0.4                       6.0.0.6 8.0.0.6 8.0.0.8 84.0.0.8                       84.0.0.4 14.14.14.14 Knowing that label 41 will be on the top of the label stack (remember the label output is read {top->bottom}) we can look for this label on P2 (the next hop according to the Explicit Route ) . P2 #show mpls forwarding label 41 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              41         28         22.22.22.22 0 [ 203 ]   \                                        5610          Et0/0      4.0.0.4     In this output we can see the local label (the label we assigned and sent upstream to the PE), the outgoing label (the label we need to use to go downstream to the next hop) and the Tunnel ID , which matches the Tunnel Instance we saw on the head-end. If we needed to find out more information about this specific tunnel from the P2 router, we can do that too. By asking P2 about the tunnel sourced from PE2 (22.22.22.22) and destined for PE4 (14.14.14.14) P2 #show mpls traffic-eng tunnel source-id 22.22.22.22 destination 14.14.14.14 P2P TUNNELS/LSPs: LSP Tunnel PE2_t0 is signalled, connection is up   InLabel  : Ethernet0/1, 41   Prev Hop : 2.0.0.12   OutLabel : Ethernet0/0, 28   Next Hop : 4.0.0.4   RSVP Signalling Info:        Src 22.22.22.22, Dst 14.14.14.14, Tun_Id 0, Tun_Instance 203     RSVP Path Info:       My Address: 4.0.0.2         Explicit Route: 4.0.0.4 6.0.0.4 6.0.0.6 8.0.0.6                       8.0.0.8 84.0.0.8 84.0.0.4 14.14.14.14 Now we continue down the path and look at the next P routers and verify label information P4 #show mpls forwarding label 28 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              28         40         22.22.22.22 0 [ 203 ]   \                                        46841         Et0/0      6.0.0.6 P6 #show mpls forwarding label 41 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              40         144        22.22.22.22 0 [ 203 ]   \                                        83918         Et0/1      6.0.0.4    P8 #show mpls forwarding label 40 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              144        Pop Label   22.22.22.22 0 [ 203 ]   \                                        47095         Et0/0      84.0.0.4    Now on P8 , the Penultimate hop, we see that our outgoing label is pop. With Traffic Engineering, just like L2 or L3VPN, we perform PHP and will pop the top label on the second to last (Penultimate) router. Now, to know what label we expect to see on PE4 , we have to think about the label stack. We started with two labels ( TE and Pseudowire labels). As we moved through the core we never added any additional labels. This means that the label P8 is removing is only the TE label, leaving the original Pseudowire label. So let's look on PE4 for the Pseudowire label PE4 #show mpls forwarding label 31 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              31         No Label   l2ckt (4096)      40483         Et0/0       point2point      Here we see label 31 points to our l2ckt (pseudowire) out Ethernet 0/0 . PE to P Tunnel Now let's look a scenario where the tunnel terminates on a P router. Again, we'll use a Pseudowire between the CE devices. When we have a TE tunnel that is not end to end and can't forward unlabeled traffic (BGP Free Core, L2/L3VPN) we must run LDP over the TE tunnel. We'll go into the "why" as soon as we start looking at the label forwarding. First, on PE1, let's look at the Pseudowire information PE1 #show mpls l2 vc 13 detail     Output interface: Tu0, imposed label stack { 23 19 17 } Unlike the previous example, we see three labels being placed on the frame. These labels (again, {Top->Bottom}) are the TE Label (Between PE1 and P1), LDP Label (between PE1 and P7), and Pseudowire Label (between PE1 and PE3). As the labeled packet arrives on P1 , it looks as the top ( TE ) label and swaps. P1 #show mpls forwarding label 23 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              23         20         11.11.11.11 0 [10]   \                                        549604        Et0/0      3.0.0.3 P3 #show mpls forwarding label 20 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              20         22         11.11.11.11 0 [10]   \                                        727533        Et0/0      5.0.0.5     Now on P5 , things get interesting. P5 is the penultimate router, so just like in our previous example P5 needs to pop the label. P5 #show mpls forwarding label 22 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              22         Pop Label 11.11.11.11 0 [10]   \                                        755760        Et0/0      7.0.0.7     Again, we think about what our label stack looked like entering P5 : { TE Label , LDP Label , Pseudowire Label }. When we pop a single label P5 we expose the LDP Label for P7 to process. Remember, since we turned LDP (mpls ip command) on the Tunnel interface we exchange an LDP label with P7 . P7 #show mpls forwarding label 19 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              19         Pop Label  13.13.13.13/32   686357        Et0/0      73.0.0.3    What would happen without this? P5 would still be the Penultimate router and would still pop the top (TE) Label. This would expose the Pseudowire and P7 wouldn't know what to do with the packet. Anytime a TE tunnel terminates on a P router you need to enable "mpls ip" on that Tunnel interface. We should also note that just like P5 , P7 is also a Penultimate router, but for a different LSP. Whereas P5 was the Penultimate router for the TE Tunnel, P7 is the Penultimate router for the Pseudowire. So, like any good Penultimate router does, P7 pops the top ( LDP ) Label. Finally, what arrives on PE3 is a single label representing the Pseudowire. PE3 #show mpls forwarding label 17 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              17         No Label   l2ckt(4096)      530983        Et0/0      point2point So the only twist in this scenario is that the TE tunnel ends on a P router, which requires us to add an additional label by enabling "mpls ip" on that tunnel interface. Out of a Tunnel and Back Into a Tunnel Now let's take a slightly different approach to the previous scenairo. Now we have the first tunnel from PE1 to P3 and a second tunnel from P3 to P7 . So our packet will leave the first TE tunnel and enter the second tunnel on the same router. Last the last two times, we look at the L2VPN information on PE1 to see our starting label stack PE1 #show mpls l2 vc det Local interface: Et0/1 up, line protocol up, Ethernet up   Destination address: 13.13.13.13, VC ID: 13, VC status: up     Output interface: Tu3, imposed label stack { 42 22 17 } Since this tunnel ends on a P router we are running LDP over it and see three labels {Tunnel LDP Pseudowire}. Since there are only three routers in the tunnel, the next hop is the Penultimate router. P1 #show mpls forwarding label 42 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              42         Pop Label   11.11.11.11 3 [1]   \                                        10209         Et0/0      3.0.0.3     P1 will pop the top (Tunnel) label and send the exposed LDP label ( 22 ) to P3. When we look at this label on P3 we now see the [T] flag set P3 #show mpls forwarding label 22    Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              22    [T]   19         13.13.13.13/32   1587549       Tu7        point2point [T]      Forwarding through a LSP tunnel.         View additional labelling info with the 'detail' option This [T] flag is set because we are entering another TE Tunnel. More importantly this LSP is inside the tunnel. This means the outgoing label we see here is not at the top of the stack. Our TE label will be added to the stack after this. We can use the detail option to see both the TE label and LDP label. P3 #show mpls forwarding label 22 detail Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              22         19         13.13.13.13/32   1597350       Tu7        point2point         MAC/Encaps=14/22, MRU=1496, Label Stack{ 22 19 }, via Et0/0         AABBCC025C10AABBCC025B008847 0001600000013000         No output feature configured P3 #show mpls traffic-eng tunnels tunnel 7 Name: P3_t7                               (Tunnel7) Destination: 7.7.7.7   Status:     Admin: up         Oper: up     Path: valid       Signalling: connected     path option 1, type dynamic (Basis for Setup, path weight 20)   InLabel  :  -   OutLabel : Ethernet0/0, 22   Next Hop : 5.0.0.5 Again, the label seen in the "show mpls forwarding" output is for the end to end LSP. This LSP gets carried inside a Traffic Engineering tunnel, so we can see this TE label with the detail option or by looking for the TE label the old-fashioned way with " show mpls traffic-eng tunnels tunnel 7 " Now P5 only looks at the top ( TE ) label and acts. P5#show mpls forwarding label 22 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              22         Pop Label  3.3.3.3 7 [2]    2242162       Et0/0      7.0.0.7     Just as we'd expect the Penultimate router to do, P5 pops the TE label sending on the original LDP label on top to P7 . Before we look at P7, let's think about the label stack on the packet leaving P5. P5 just removed the TE label leaving the P3 <---> P7 LDP label. Under this we should still have the PE1 <---> PE3 Pseudowire label. P7 sees the P3 LDP label (the one with the [T] flag earlier) P7 #show mpls forwarding label 19 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              19         Pop Label  13.13.13.13/32   29952955      Et0/0      73.0.0.3    This exposes the Pseudowire label to be processed by PE3 PE3 #show mpls forwarding label 17 Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              17         No Label   l2ckt(4096)      22508357      Et0/0      point2point Hopefully this paints a picture of how label operations work in various TE tunnel topologies. Fast Reroute scenarios will be covered in the next chapter, coming soon! ... View more

  The Problem On traditional switches whenever we have a trunk interface we use the VLAN tag to demultiplex the VLANs. The switch needs to determine which MAC Address table to look in for a forwarding decision. To do this we require the switch to do two things: 1.) Have the VLAN configured globally 2.) Perform MAC learning in this VLAN The challenge with this is that it requires us to use finite resources, perhaps without reason. Since the 802.1q VLAN tag is only 12-bits wide we can only configure a maximum of 4096 VLANs. Furthermore switches have a finite amount of CAM space for MAC Learning limiting the number of hosts we can support.  In modern provider and cloud environments there is a need to scale beyond these limitations. Enter EVCs Ethernet Virtual Circuits (EVCs) allow us to leverage existing 802.1q VLAN tags in a brand new way. Traditionally the VLAN tag defined both classification (which VLAN) and forwarding (which CAM table to do a MAC lookup in). Now, with EVCs we can separate these concepts; the VLAN tag is used for classification and the Service Instance defines the forwarding action. For example we could allocate VLAN 10 to different customers on every switchport and forward each customer's traffic across different MPLS Pseudowires, but never actually configure VLAN 10 globally! How It Works (Ingress) As an 802.1q tagged frame enters an interface that has been configured with an EVC we will determine which EVC it is classified into based on the tags on the frame. Within the EVC we define what action we wish to do with that frame. Let's take a look at a sample EVC configuration. interface GigabitEthernet0/2 switchport trunk allowed vlan none  switchport mode trunk service instance 6 ethernet   encapsulation dot1q 10   rewrite ingress tag pop 1 symmetric   xconnect 192.168.1.1 33 encapsulation mpls Now, let's break down each piece of this configuration.  switchport trunk allowed vlan none  switchport mode trunk These lines tell the interface that we need to process 802.1q tags as a trunk interface should, however we will not actually pass any VLANs through this interface. VLAN tags received on Service Instance interfaces have no direct relationship to VLANs configured on the switch. Furthermore Service Instance interfaces do not do any MAC learning (except through a bridge-domain VLAN interface, which is discussed later). Because of this we do not want to allow any globally configured VLANs across this trunk interface. We only need to enable VLAN tag processing and let the Service Instance figure out what to do with the frame. service instance 6 ethernet This defines the service instance. The number is arbitrary; it has nothing to do with the VLANs that will be processed by this particular Service Instance The "ethernet" keyword is always used. encapsulation dot1q 10 This is how we map an incoming tag to a service instance. If VLAN tag 10 is received on this interface it will be put into service instance 6 . rewrite ingress tag pop 1 symmetric Since the incoming tag no longer has any inherent meaning beyond this specific interface we need a way to discard that tag before forwarding the frame on. The rewrite ingress command does just that. In this case we will remove exactly 1 tag, This command is optional and there are a number options that can be done beyond simply removing the tag including, VLAN translation and imposing additional tags. We will discuss some of these options and the " symmetric " keyword a little later. Finally, what is our forwarding action with that frame? xconnect 192.168.1.1 33 encapsulation mpls This tells us that the frame should be sent across the L2VPN MPLS cloud. Since before this we configure the rewrite ingress tag pop 1 symmetric command we will send a frame with no VLAN Tags across the MPLS pseudowire. There are a multiple possible forwarding actions including routing and local switching (connect) How It Works (Egress) As a frame is received it will be classified to go out the Service Instance interface based on how it arrived on this switch in the first place. For example if the frame is received across MPLS pseudowire 33, we automatically know it is part of service instance 6 Continuing to work bottom up in the configuration we come to the symmetric part of rewrite ingress tag pop 1 symmetric . Since we popped 1 tag ingress, to be symmetric we need to push 1 tag egress. The symmetric keyword will always be used with the rewrite command. We determine which tag to impose based on the encapsulation dot1q 10 command. As the frame egresses the interface impose VLAN Tag of 10. Keep in mind that the access layer device that is sending us tagged frames is most likely a traditional Layer-2 switch and needs the tag it sends to be the same tag it receives for proper classification.     Step-by-Step Example Here's a sample topology, with two access switches processing different VLANs. The service instance configurations are on PE Blue and PE Purple.   So a few things to note here: The access layer switches are sending and expecting different VLAN tags. The service instance numbers are arbitrary The VLAN tag will be popped before being sent into the MPLS cloud A client connect to the access port will send an untagged frame. This frame will be have VLAN tag 10 added to it by the access layer switch and sent to the PE with the service instance configuration. The Blue PE will see VLAN tag 10 and place it into service instance 9. Since we have configured the 'rewrite ingress pop 1 symmetric' command, we will pop the first tag before applying an MPLS label and forwarding into the MPLS cloud.   As the labeled packet leaves the MPLS cloud we place the untagged frame into PE Red's service instance 18, based on the "xconnect" command. From there since the "rewrite ingress pop 1 symmetric" command is configured and this is an egress frame we know we need to impose one. The tag imposed is based on the "encapsulation dot1q" configuration, so in this case, VLAN tag 11 is imposed on the frame before sending back out to the access layer switch. EVC Options Flexible Matching One of the things that make EVCs so powerful is their flexible matching criteria. EVCs allow us to classify inbound frames in a highly flexible manner based on 1 or more VLAN tags or CoS values. Here are some examples   Configuration Effect encapsulation dot1q 10 Match the single VLAN tag 10 encapsulation dot1q 25 second-dot1q 13 Match first VLAN tag 25 and second tag 13 encapsulation dot1q any second-dot1q 22 Match any double tagged frame with a second tag of 22 encapsulation dot1q 16 cos 4 Match a single tag 16 when it has CoS value 4 encapsulation dot1q untagged Match the native (untagged) VLAN encapsulation dot1q default The catch all class for all traffic not previously classified   The options here are not exhaustive but just some examples. The other thing to remember about tag matching is that we follow a longest match criteria.   Rewrite Options Along with a number of flexible matching options we have numerous tag rewrite options.   Configuration Effect rewrite ingress tag pop 1 symmetric remove the top 802.1q tag rewrite ingress tag pop 2 symmetric remove the top two 802.1q tags rewrite ingress tag translate 1-to-1 dot1q  28 symmetric remove the top tag and replace it with 28 rewrite ingress tag translate 2-to-2 dot1 22 second-dot1q 23 remove the top two tags and replace them with 22 and 23 (23 will be the inner tag) rewrite ingress tag push dot1q 56 second-dot1q 55 push two new tags on top of the existing frame. The top tag will be 56; inner tag of 55   Forwarding Options An EVC can be attached to an MPLS xconnect and we can send the traffic across an MPLS cloud. For more flexibility EVCs introduce the concept of the Bridge Domain . A Bridge Domain is what is traditionally thought of as a Layer 3 SVI. Unlike the VLAN tags that are being processed by the configured EVCs bridge-domains do require the VLAN to be configured globally on the device and use platform wide resources.  Here is an example of an interface configured with a bridge-domain:  interface g0/2   service instance 1 ethernet     encapsulation dot1q 18     rewrite ingres tag pop 1 symmetric     bridge-domain 44 ! interface Vlan44   ip address 192.168.1.1 255.255.255.0   The packet, without VLAN tags, will be passed to the VLAN44 interface for normal routing to occur. We can also tie multiple service instances to the same bridge-domain to make forwarding tagged traffic highly flexible.   interface g0/2   service instance 1 ethernet     encapsulation dot1q 18     rewrite ingres tag pop 1 symmetric     bridge-domain 44   service instance 2 ethernet     encapsulation dot1q 66     rewrite ingress tag pop 1 symmetric    bridge-domain 44 ! interface Vlan44   ip address 192.168.1.1 255.255.255.0   This configuration will allow either the service instances to speak between one another or out to other routed subnets. More broadly, Vlan 44 will bridge together the two service instances and forward frames based on MAC learning. As a frame enters Service Instance 1, the VLAN tag will be removed, the frame will be passed to Vlan 44 where the destination MAC will be looked up. If the destination is out the other Service Instance, the frame will be placed on Service Instance 2 and a new VLAN tag will be added.  Bridge domains also allow for the configuration of a " split-horizon " (bridge-domain 44 split-horizon) to prevent inter-EVC communication, only allowing for routing outside of the bridge domain. Finally, we can take multiple EVCs, and send all of them over an MPLS Pseudowire interface g0/2   service instance 1 ethernet     encapsulation dot1q 18     rewrite ingres tag pop 1 symmetric     bridge-domain 44 split-horizon   service instance 2 ethernet     encapsulation dot1q 66     rewrite ingress tag pop 1 symmetric     bridge-domain 44 split-horizon ! interface Vlan44   xconnect 192.168.1.1 55 encapsulation mpls   This will allow us to take two different VLANs and send them to the same MPLS endpoint, removing the VLAN tags in the process. In this scenario we will learn MAC address on both service instances and send them both over the single xconnect, but we prevent traffic on service instance 2 from being sent to service instance 1. Final Thoughts Since the way EVCs work is so different from traditional switching not all switching platforms are capable of doing the EVC frame manipulation independently of the forwarding action. Newer platforms like the me3600x or me3800x were designed from the ground up with this kind of capability in mind. The 7600 platform requires newer Ethernet Services (ES) modules to do the additional work that the Supervisor and DFC forwarding engines are unable to do. This guide is also not an exhaustive list of supported platforms or configurations, but merely to demonstrate some deployment options and how traffic forwarding operates in these new EVC environments. ... View more