The Problem
On traditional switches whenever we have a trunk interface we use the VLAN tag to demultiplex the VLANs. The switch needs to determine which MAC Address table to look in for a forwarding decision. To do this we require the switch to do two things:
1.) Have the VLAN configured globally
2.) Perform MAC learning in this VLAN
The challenge with this is that it requires us to use finite resources, perhaps without reason. Since the 802.1q VLAN tag is only 12-bits wide we can only configure a maximum of 4096 VLANs. Furthermore switches have a finite amount of CAM space for MAC Learning limiting the number of hosts we can support.
In modern provider and cloud environments there is a need to scale beyond these limitations.
Enter EVCs
Ethernet Virtual Circuits (EVCs) allow us to leverage existing 802.1q VLAN tags in a brand new way. Traditionally the VLAN tag defined both classification (which VLAN) and forwarding (which CAM table to do a MAC lookup in). Now, with EVCs we can separate these concepts; the VLAN tag is used for classification and the Service Instance defines the forwarding action. For example we could allocate VLAN 10 to different customers on every switchport and forward each customer's traffic across different MPLS Pseudowires, but never actually configure VLAN 10 globally!
How It Works (Ingress)
As an 802.1q tagged frame enters an interface that has been configured with an EVC we will determine which EVC it is classified into based on the tags on the frame. Within the EVC we define what action we wish to do with that frame. Let's take a look at a sample EVC configuration.
interface GigabitEthernet0/2
switchport trunk allowed vlan none
switchport mode trunk
service instance 6 ethernet
encapsulation dot1q 10
rewrite ingress tag pop 1 symmetric
xconnect 192.168.1.1 33 encapsulation mpls
Now, let's break down each piece of this configuration.
switchport trunk allowed vlan none
switchport mode trunk
These lines tell the interface that we need to process 802.1q tags as a trunk interface should, however we will not actually pass any VLANs through this interface. VLAN tags received on Service Instance interfaces have no direct relationship to VLANs configured on the switch. Furthermore Service Instance interfaces do not do any MAC learning (except through a bridge-domain VLAN interface, which is discussed later). Because of this we do not want to allow any globally configured VLANs across this trunk interface. We only need to enable VLAN tag processing and let the Service Instance figure out what to do with the frame.
service instance 6 ethernet
This defines the service instance. The number is arbitrary; it has nothing to do with the VLANs that will be processed by this particular Service Instance The "ethernet" keyword is always used.
encapsulation dot1q 10
This is how we map an incoming tag to a service instance. If VLAN tag 10 is received on this interface it will be put into service instance 6.
rewrite ingress tag pop 1 symmetric
Since the incoming tag no longer has any inherent meaning beyond this specific interface we need a way to discard that tag before forwarding the frame on. The rewrite ingress command does just that. In this case we will remove exactly 1 tag, This command is optional and there are a number options that can be done beyond simply removing the tag including, VLAN translation and imposing additional tags. We will discuss some of these options and the "symmetric" keyword a little later.
Finally, what is our forwarding action with that frame?
xconnect 192.168.1.1 33 encapsulation mpls
This tells us that the frame should be sent across the L2VPN MPLS cloud. Since before this we configure the rewrite ingress tag pop 1 symmetric command we will send a frame with no VLAN Tags across the MPLS pseudowire. There are a multiple possible forwarding actions including routing and local switching (connect)
How It Works (Egress)
As a frame is received it will be classified to go out the Service Instance interface based on how it arrived on this switch in the first place. For example if the frame is received across MPLS pseudowire 33, we automatically know it is part of service instance 6
Continuing to work bottom up in the configuration we come to the symmetric part of rewrite ingress tag pop 1 symmetric. Since we popped 1 tag ingress, to be symmetric we need to push 1 tag egress. The symmetric keyword will always be used with the rewrite command.
We determine which tag to impose based on the encapsulation dot1q 10 command. As the frame egresses the interface impose VLAN Tag of 10. Keep in mind that the access layer device that is sending us tagged frames is most likely a traditional Layer-2 switch and needs the tag it sends to be the same tag it receives for proper classification.
Step-by-Step Example
Here's a sample topology, with two access switches processing different VLANs. The service instance configurations are on PE Blue and PE Purple.
So a few things to note here:
- The access layer switches are sending and expecting different VLAN tags.
- The service instance numbers are arbitrary
- The VLAN tag will be popped before being sent into the MPLS cloud
A client connect to the access port will send an untagged frame. This frame will be have VLAN tag 10 added to it by the access layer switch and sent to the PE with the service instance configuration.
The Blue PE will see VLAN tag 10 and place it into service instance 9. Since we have configured the 'rewrite ingress pop 1 symmetric' command, we will pop the first tag before applying an MPLS label and forwarding into the MPLS cloud.
As the labeled packet leaves the MPLS cloud we place the untagged frame into PE Red's service instance 18, based on the "xconnect" command. From there since the "rewrite ingress pop 1 symmetric" command is configured and this is an egress frame we know we need to impose one. The tag imposed is based on the "encapsulation dot1q" configuration, so in this case, VLAN tag 11 is imposed on the frame before sending back out to the access layer switch.
EVC Options
Flexible Matching
One of the things that make EVCs so powerful is their flexible matching criteria. EVCs allow us to classify inbound frames in a highly flexible manner based on 1 or more VLAN tags or CoS values. Here are some examples
ConfigurationEffect
| encapsulation dot1q 10 | Match the single VLAN tag 10 |
encapsulation dot1q 25 second-dot1q 13 | Match first VLAN tag 25 and second tag 13 |
| encapsulation dot1q any second-dot1q 22 | Match any double tagged frame with a second tag of 22 |
encapsulation dot1q 16 cos 4 | Match a single tag 16 when it has CoS value 4 |
| encapsulation dot1q untagged | Match the native (untagged) VLAN |
| encapsulation dot1q default | The catch all class for all traffic not previously classified |
The options here are not exhaustive but just some examples. The other thing to remember about tag matching is that we follow a longest match criteria.
Rewrite Options
Along with a number of flexible matching options we have numerous tag rewrite options.
ConfigurationEffect
| rewrite ingress tag pop 1 symmetric | remove the top 802.1q tag |
| rewrite ingress tag pop 2 symmetric | remove the top two 802.1q tags |
rewrite ingress tag translate 1-to-1 dot1q 28 symmetric | remove the top tag and replace it with 28 |
rewrite ingress tag translate 2-to-2 dot1 22 second-dot1q 23 | remove the top two tags and replace them with 22 and 23 (23 will be the inner tag) |
| rewrite ingress tag push dot1q 56 second-dot1q 55 | push two new tags on top of the existing frame. The top tag will be 56; inner tag of 55 |
Forwarding Options
An EVC can be attached to an MPLS xconnect and we can send the traffic across an MPLS cloud.
For more flexibility EVCs introduce the concept of the Bridge Domain. A Bridge Domain is what is traditionally thought of as a Layer 3 SVI. Unlike the VLAN tags that are being processed by the configured EVCs bridge-domains do require the VLAN to be configured globally on the device and use platform wide resources.
Here is an example of an interface configured with a bridge-domain:
interface g0/2
service instance 1 ethernet
encapsulation dot1q 18
rewrite ingres tag pop 1 symmetric
bridge-domain 44
!
interface Vlan44
ip address 192.168.1.1 255.255.255.0
The packet, without VLAN tags, will be passed to the VLAN44 interface for normal routing to occur.
We can also tie multiple service instances to the same bridge-domain to make forwarding tagged traffic highly flexible.
interface g0/2
service instance 1 ethernet
encapsulation dot1q 18
rewrite ingres tag pop 1 symmetric
bridge-domain 44
service instance 2 ethernet
encapsulation dot1q 66
rewrite ingress tag pop 1 symmetric
bridge-domain 44
!
interface Vlan44
ip address 192.168.1.1 255.255.255.0
This configuration will allow either the service instances to speak between one another or out to other routed subnets. More broadly, Vlan 44 will bridge together the two service instances and forward frames based on MAC learning. As a frame enters Service Instance 1, the VLAN tag will be removed, the frame will be passed to Vlan 44 where the destination MAC will be looked up. If the destination is out the other Service Instance, the frame will be placed on Service Instance 2 and a new VLAN tag will be added.
Bridge domains also allow for the configuration of a "split-horizon" (bridge-domain 44 split-horizon) to prevent inter-EVC communication, only allowing for routing outside of the bridge domain.
Finally, we can take multiple EVCs, and send all of them over an MPLS Pseudowire
interface g0/2
service instance 1 ethernet
encapsulation dot1q 18
rewrite ingres tag pop 1 symmetric
bridge-domain 44 split-horizon
service instance 2 ethernet
encapsulation dot1q 66
rewrite ingress tag pop 1 symmetric
bridge-domain 44 split-horizon
!
interface Vlan44
xconnect 192.168.1.1 55 encapsulation mpls
This will allow us to take two different VLANs and send them to the same MPLS endpoint, removing the VLAN tags in the process. In this scenario we will learn MAC address on both service instances and send them both over the single xconnect, but we prevent traffic on service instance 2 from being sent to service instance 1.
Final Thoughts
Since the way EVCs work is so different from traditional switching not all switching platforms are capable of doing the EVC frame manipulation independently of the forwarding action. Newer platforms like the me3600x or me3800x were designed from the ground up with this kind of capability in mind. The 7600 platform requires newer Ethernet Services (ES) modules to do the additional work that the Supervisor and DFC forwarding engines are unable to do. This guide is also not an exhaustive list of supported platforms or configurations, but merely to demonstrate some deployment options and how traffic forwarding operates in these new EVC environments.
great doc
@plumbis wrote:
The Problem
On traditional switches whenever we have a trunk interface we use the VLAN tag to demultiplex the VLANs. The switch needs to determine which MAC Address table to look in for a forwarding decision. To do this we require the switch to do two things:
1.) Have the VLAN configured globally
2.) Perform MAC learning in this VLAN
The challenge with this is that it requires us to use finite resources, perhaps without reason. Since the 802.1q VLAN tag is only 12-bits wide we can only configure a maximum of 4096 VLANs. Furthermore switches have a finite amount of CAM space for MAC Learning limiting the number of hosts we can support.
In modern provider and cloud environments there is a need to scale beyond these limitations.
Enter EVCs
Ethernet Virtual Circuits (EVCs) allow us to leverage existing 802.1q VLAN tags in a brand new way. Traditionally the VLAN tag defined both classification (which VLAN) and forwarding (which CAM table to do a MAC lookup in). Now, with EVCs we can separate these concepts; the VLAN tag is used for classification and the Service Instance defines the forwarding action. For example we could allocate VLAN 10 to different customers on every switchport and forward each customer's traffic across different MPLS Pseudowires, but never actually configure VLAN 10 globally!
How It Works (Ingress)
As an 802.1q tagged frame enters an interface that has been configured with an EVC we will determine which EVC it is classified into based on the tags on the frame. Within the EVC we define what action we wish to do with that frame. Let's take a look at a sample EVC configuration.
interface GigabitEthernet0/2
switchport trunk allowed vlan none
switchport mode trunk
service instance 6 ethernet
encapsulation dot1q 10
rewrite ingress tag pop 1 symmetric
xconnect 192.168.1.1 33 encapsulation mpls
Now, let's break down each piece of this configuration.
switchport trunk allowed vlan none
switchport mode trunk
These lines tell the interface that we need to process 802.1q tags as a trunk interface should, however we will not actually pass any VLANs through this interface. VLAN tags received on Service Instance interfaces have no direct relationship to VLANs configured on the switch. Furthermore Service Instance interfaces do not do any MAC learning (except through a bridge-domain VLAN interface, which is discussed later). Because of this we do not want to allow any globally configured VLANs across this trunk interface. We only need to enable VLAN tag processing and let the Service Instance figure out what to do with the frame.
service instance 6 ethernet
This defines the service instance. The number is arbitrary; it has nothing to do with the VLANs that will be processed by this particular Service Instance The "ethernet" keyword is always used.
encapsulation dot1q 10
This is how we map an incoming tag to a service instance. If VLAN tag 10 is received on this interface it will be put into service instance 6.
rewrite ingress tag pop 1 symmetric
Since the incoming tag no longer has any inherent meaning beyond this specific interface we need a way to discard that tag before forwarding the frame on. The rewrite ingress command does just that. In this case we will remove exactly 1 tag, This command is optional and there are a number options that can be done beyond simply removing the tag including, VLAN translation and imposing additional tags. We will discuss some of these options and the "symmetric" keyword a little later.
Finally, what is our forwarding action with that frame?
xconnect 192.168.1.1 33 encapsulation mpls
This tells us that the frame should be sent across the L2VPN MPLS cloud. Since before this we configure the rewrite ingress tag pop 1 symmetric command we will send a frame with no VLAN Tags across the MPLS pseudowire. There are a multiple possible forwarding actions including routing and local switching (connect)
How It Works (Egress)
As a frame is received it will be classified to go out the Service Instance interface based on how it arrived on this switch in the first place. For example if the frame is received across MPLS pseudowire 33, we automatically know it is part of service instance 6
Continuing to work bottom up in the configuration we come to the symmetric part of rewrite ingress tag pop 1 symmetric. Since we popped 1 tag ingress, to be symmetric we need to push 1 tag egress. The symmetric keyword will always be used with the rewrite command.
We determine which tag to impose based on the encapsulation dot1q 10 command. As the frame egresses the interface impose VLAN Tag of 10. Keep in mind that the access layer device that is sending us tagged frames is most likely a traditional Layer-2 switch and needs the tag it sends to be the same tag it receives for proper classification.
Step-by-Step Example
Here's a sample topology, with two access switches processing different VLANs. The service instance configurations are on PE Blue and PE Purple.
So a few things to note here:
- The access layer switches are sending and expecting different VLAN tags.
- The service instance numbers are arbitrary
- The VLAN tag will be popped before being sent into the MPLS cloud
A client connect to the access port will send an untagged frame. This frame will be have VLAN tag 10 added to it by the access layer switch and sent to the PE with the service instance configuration.
The Blue PE will see VLAN tag 10 and place it into service instance 9. Since we have configured the 'rewrite ingress pop 1 symmetric' command, we will pop the first tag before applying an MPLS label and forwarding into the MPLS cloud.
As the labeled packet leaves the MPLS cloud we place the untagged frame into PE Red's service instance 18, based on the "xconnect" command. From there since the "rewrite ingress pop 1 symmetric" command is configured and this is an egress frame we know we need to impose one. The tag imposed is based on the "encapsulation dot1q" configuration, so in this case, VLAN tag 11 is imposed on the frame before sending back out to the access layer switch.
EVC Options
Flexible Matching
One of the things that make EVCs so powerful is their flexible matching criteria. EVCs allow us to classify inbound frames in a highly flexible manner based on 1 or more VLAN tags or CoS values. Here are some examples
ConfigurationEffect
encapsulation dot1q 10 Match the single VLAN tag 10 encapsulation dot1q 25 second-dot1q 13
Match first VLAN tag 25 and second tag 13 encapsulation dot1q any second-dot1q 22 Match any double tagged frame with a second tag of 22 encapsulation dot1q 16 cos 4
Match a single tag 16 when it has CoS value 4 encapsulation dot1q untagged Match the native (untagged) VLAN encapsulation dot1q default The catch all class for all traffic not previously classified
The options here are not exhaustive but just some examples. The other thing to remember about tag matching is that we follow a longest match criteria.
Rewrite Options
Along with a number of flexible matching options we have numerous tag rewrite options.
ConfigurationEffect
rewrite ingress tag pop 1 symmetric remove the top 802.1q tag rewrite ingress tag pop 2 symmetric remove the top two 802.1q tags rewrite ingress tag translate 1-to-1 dot1q 28 symmetric
remove the top tag and replace it with 28 rewrite ingress tag translate 2-to-2 dot1 22 second-dot1q 23
remove the top two tags and replace them with 22 and 23 (23 will be the inner tag) rewrite ingress tag push dot1q 56 second-dot1q 55 push two new tags on top of the existing frame. The top tag will be 56; inner tag of 55
Forwarding Options
An EVC can be attached to an MPLS xconnect and we can send the traffic across an MPLS cloud.
For more flexibility EVCs introduce the concept of the Bridge Domain. A Bridge Domain is what is traditionally thought of as a Layer 3 SVI. Unlike the VLAN tags that are being processed by the configured EVCs bridge-domains do require the VLAN to be configured globally on the device and use platform wide resources.
Here is an example of an interface configured with a bridge-domain:
interface g0/2
service instance 1 ethernet
encapsulation dot1q 18
rewrite ingres tag pop 1 symmetric
bridge-domain 44
!
interface Vlan44
ip address 192.168.1.1 255.255.255.0
The packet, without VLAN tags, will be passed to the VLAN44 interface for normal routing to occur.
We can also tie multiple service instances to the same bridge-domain to make forwarding tagged traffic highly flexible.
interface g0/2
service instance 1 ethernet
encapsulation dot1q 18
rewrite ingres tag pop 1 symmetric
bridge-domain 44
service instance 2 ethernet
encapsulation dot1q 66
rewrite ingress tag pop 1 symmetric
bridge-domain 44
!
interface Vlan44
ip address 192.168.1.1 255.255.255.0
This configuration will allow either the service instances to speak between one another or out to other routed subnets. More broadly, Vlan 44 will bridge together the two service instances and forward frames based on MAC learning. As a frame enters Service Instance 1, the VLAN tag will be removed, the frame will be passed to Vlan 44 where the destination MAC will be looked up. If the destination is out the other Service Instance, the frame will be placed on Service Instance 2 and a new VLAN tag will be added.
Bridge domains also allow for the configuration of a "split-horizon" (bridge-domain 44 split-horizon) to prevent inter-EVC communication, only allowing for routing outside of the bridge domain.
Finally, we can take multiple EVCs, and send all of them over an MPLS Pseudowire
interface g0/2
service instance 1 ethernet
encapsulation dot1q 18
rewrite ingres tag pop 1 symmetric
bridge-domain 44 split-horizon
service instance 2 ethernet
encapsulation dot1q 66
rewrite ingress tag pop 1 symmetric
bridge-domain 44 split-horizon
!
interface Vlan44
xconnect 192.168.1.1 55 encapsulation mpls
This will allow us to take two different VLANs and send them to the same MPLS endpoint, removing the VLAN tags in the process. In this scenario we will learn MAC address on both service instances and send them both over the single xconnect, but we prevent traffic on service instance 2 from being sent to service instance 1.
Final Thoughts
Since the way EVCs work is so different from traditional switching not all switching platforms are capable of doing the EVC frame manipulation independently of the forwarding action. Newer platforms like the me3600x or me3800x were designed from the ground up with this kind of capability in mind. The 7600 platform requires newer Ethernet Services (ES) modules to do the additional work that the Supervisor and DFC forwarding engines are unable to do. This guide is also not an exhaustive list of supported platforms or configurations, but merely to demonstrate some deployment options and how traffic forwarding operates in these new EVC environments.
