cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
526751
Views
380
Helpful
66
Comments
plumbis
Level 7
Level 7

 

The Problem

On traditional switches whenever we have a trunk interface we use the VLAN tag to demultiplex the VLANs. The switch needs to determine which MAC Address table to look in for a forwarding decision. To do this we require the switch to do two things:

1.) Have the VLAN configured globally

2.) Perform MAC learning in this VLAN

The challenge with this is that it requires us to use finite resources, perhaps without reason. Since the 802.1q VLAN tag is only 12-bits wide we can only configure a maximum of 4096 VLANs. Furthermore switches have a finite amount of CAM space for MAC Learning limiting the number of hosts we can support. 

In modern provider and cloud environments there is a need to scale beyond these limitations.

Enter EVCs

Ethernet Virtual Circuits (EVCs) allow us to leverage existing 802.1q VLAN tags in a brand new way. Traditionally the VLAN tag defined both classification (which VLAN) and forwarding (which CAM table to do a MAC lookup in). Now, with EVCs we can separate these concepts; the VLAN tag is used for classification and the Service Instance defines the forwarding action. For example we could allocate VLAN 10 to different customers on every switchport and forward each customer's traffic across different MPLS Pseudowires, but never actually configure VLAN 10 globally!

How It Works (Ingress)

As an 802.1q tagged frame enters an interface that has been configured with an EVC we will determine which EVC it is classified into based on the tags on the frame. Within the EVC we define what action we wish to do with that frame. Let's take a look at a sample EVC configuration.

interface GigabitEthernet0/2

switchport trunk allowed vlan none 

switchport mode trunk

service instance 6 ethernet

  encapsulation dot1q 10

  rewrite ingress tag pop 1 symmetric

  xconnect 192.168.1.1 33 encapsulation mpls

Now, let's break down each piece of this configuration. 

switchport trunk allowed vlan none 

switchport mode trunk

These lines tell the interface that we need to process 802.1q tags as a trunk interface should, however we will not actually pass any VLANs through this interface. VLAN tags received on Service Instance interfaces have no direct relationship to VLANs configured on the switch. Furthermore Service Instance interfaces do not do any MAC learning (except through a bridge-domain VLAN interface, which is discussed later). Because of this we do not want to allow any globally configured VLANs across this trunk interface. We only need to enable VLAN tag processing and let the Service Instance figure out what to do with the frame.

service instance 6 ethernet

This defines the service instance. The number is arbitrary; it has nothing to do with the VLANs that will be processed by this particular Service Instance The "ethernet" keyword is always used.

encapsulation dot1q 10

This is how we map an incoming tag to a service instance. If VLAN tag 10 is received on this interface it will be put into service instance 6.

rewrite ingress tag pop 1 symmetric

Since the incoming tag no longer has any inherent meaning beyond this specific interface we need a way to discard that tag before forwarding the frame on. The rewrite ingress command does just that. In this case we will remove exactly 1 tag, This command is optional and there are a number options that can be done beyond simply removing the tag including, VLAN translation and imposing additional tags. We will discuss some of these options and the "symmetric" keyword a little later.

Finally, what is our forwarding action with that frame?

xconnect 192.168.1.1 33 encapsulation mpls

This tells us that the frame should be sent across the L2VPN MPLS cloud. Since before this we configure the rewrite ingress tag pop 1 symmetric command we will send a frame with no VLAN Tags across the MPLS pseudowire. There are a multiple possible forwarding actions including routing and local switching (connect)

How It Works (Egress)

As a frame is received it will be classified to go out the Service Instance interface based on how it arrived on this switch in the first place. For example if the frame is received across MPLS pseudowire 33, we automatically know it is part of service instance 6

Continuing to work bottom up in the configuration we come to the symmetric part of rewrite ingress tag pop 1 symmetric. Since we popped 1 tag ingress, to be symmetric we need to push 1 tag egress. The symmetric keyword will always be used with the rewrite command.

We determine which tag to impose based on the encapsulation dot1q 10 command. As the frame egresses the interface impose VLAN Tag of 10. Keep in mind that the access layer device that is sending us tagged frames is most likely a traditional Layer-2 switch and needs the tag it sends to be the same tag it receives for proper classification.

 

Flow Direction.png

 

Step-by-Step Example

Here's a sample topology, with two access switches processing different VLANs. The service instance configurations are on PE Blue and PE Purple.

 

Example Configuration.png

So a few things to note here:

  • The access layer switches are sending and expecting different VLAN tags.
  • The service instance numbers are arbitrary
  • The VLAN tag will be popped before being sent into the MPLS cloud

Frame Example 1.png

A client connect to the access port will send an untagged frame. This frame will be have VLAN tag 10 added to it by the access layer switch and sent to the PE with the service instance configuration.

The Blue PE will see VLAN tag 10 and place it into service instance 9. Since we have configured the 'rewrite ingress pop 1 symmetric' command, we will pop the first tag before applying an MPLS label and forwarding into the MPLS cloud.

Frame Example 2.png

 

As the labeled packet leaves the MPLS cloud we place the untagged frame into PE Red's service instance 18, based on the "xconnect" command. From there since the "rewrite ingress pop 1 symmetric" command is configured and this is an egress frame we know we need to impose one. The tag imposed is based on the "encapsulation dot1q" configuration, so in this case, VLAN tag 11 is imposed on the frame before sending back out to the access layer switch.

EVC Options

Flexible Matching

One of the things that make EVCs so powerful is their flexible matching criteria. EVCs allow us to classify inbound frames in a highly flexible manner based on 1 or more VLAN tags or CoS values. Here are some examples

 

ConfigurationEffect

encapsulation dot1q 10Match the single VLAN tag 10

encapsulation dot1q 25 second-dot1q 13

Match first VLAN tag 25 and second tag 13
encapsulation dot1q any second-dot1q 22Match any double tagged frame with a second tag of 22

encapsulation dot1q 16 cos 4

Match a single tag 16 when it has CoS value 4
encapsulation dot1q untaggedMatch the native (untagged) VLAN
encapsulation dot1q defaultThe catch all class for all traffic not previously classified

 

The options here are not exhaustive but just some examples. The other thing to remember about tag matching is that we follow a longest match criteria.

 

Rewrite Options

Along with a number of flexible matching options we have numerous tag rewrite options.

 

ConfigurationEffect

rewrite ingress tag pop 1 symmetricremove the top 802.1q tag
rewrite ingress tag pop 2 symmetricremove the top two 802.1q tags

rewrite ingress tag translate 1-to-1 dot1q  28 symmetric

remove the top tag and replace it with 28

rewrite ingress tag translate 2-to-2 dot1 22 second-dot1q 23

remove the top two tags and replace them with 22 and 23 (23 will be the inner tag)
rewrite ingress tag push dot1q 56 second-dot1q 55push two new tags on top of the existing frame. The top tag will be 56; inner tag of 55

 

Forwarding Options

An EVC can be attached to an MPLS xconnect and we can send the traffic across an MPLS cloud.

For more flexibility EVCs introduce the concept of the Bridge Domain. A Bridge Domain is what is traditionally thought of as a Layer 3 SVI. Unlike the VLAN tags that are being processed by the configured EVCs bridge-domains do require the VLAN to be configured globally on the device and use platform wide resources. 

Here is an example of an interface configured with a bridge-domain: 

interface g0/2

  service instance 1 ethernet

    encapsulation dot1q 18

    rewrite ingres tag pop 1 symmetric

    bridge-domain 44

!

interface Vlan44

  ip address 192.168.1.1 255.255.255.0

 

The packet, without VLAN tags, will be passed to the VLAN44 interface for normal routing to occur.

We can also tie multiple service instances to the same bridge-domain to make forwarding tagged traffic highly flexible.

 

interface g0/2

  service instance 1 ethernet

    encapsulation dot1q 18

    rewrite ingres tag pop 1 symmetric

    bridge-domain 44

  service instance 2 ethernet

    encapsulation dot1q 66

    rewrite ingress tag pop 1 symmetric

   bridge-domain 44

!

interface Vlan44

  ip address 192.168.1.1 255.255.255.0

 

This configuration will allow either the service instances to speak between one another or out to other routed subnets. More broadly, Vlan 44 will bridge together the two service instances and forward frames based on MAC learning. As a frame enters Service Instance 1, the VLAN tag will be removed, the frame will be passed to Vlan 44 where the destination MAC will be looked up. If the destination is out the other Service Instance, the frame will be placed on Service Instance 2 and a new VLAN tag will be added. 

Bridge domains also allow for the configuration of a "split-horizon" (bridge-domain 44 split-horizon) to prevent inter-EVC communication, only allowing for routing outside of the bridge domain.

Finally, we can take multiple EVCs, and send all of them over an MPLS Pseudowire

interface g0/2

  service instance 1 ethernet

    encapsulation dot1q 18

    rewrite ingres tag pop 1 symmetric

    bridge-domain 44 split-horizon

  service instance 2 ethernet

    encapsulation dot1q 66

    rewrite ingress tag pop 1 symmetric

    bridge-domain 44 split-horizon

!

interface Vlan44

  xconnect 192.168.1.1 55 encapsulation mpls

 

This will allow us to take two different VLANs and send them to the same MPLS endpoint, removing the VLAN tags in the process. In this scenario we will learn MAC address on both service instances and send them both over the single xconnect, but we prevent traffic on service instance 2 from being sent to service instance 1.

Final Thoughts

Since the way EVCs work is so different from traditional switching not all switching platforms are capable of doing the EVC frame manipulation independently of the forwarding action. Newer platforms like the me3600x or me3800x were designed from the ground up with this kind of capability in mind. The 7600 platform requires newer Ethernet Services (ES) modules to do the additional work that the Supervisor and DFC forwarding engines are unable to do. This guide is also not an exhaustive list of supported platforms or configurations, but merely to demonstrate some deployment options and how traffic forwarding operates in these new EVC environments.

Comments
mitovradoslav
Community Member

Very well written, Thank you!

Good day

Is there any limitation in a solution  of Storage as a Service if I have a xconnect deploy for conection between the client side and the provider cloud ? or I have to meet some requirement ?

Regards

Wilson

Odys (CSC)
Level 1
Level 1

A great presentation. I've learned a lot from it.

It's been hard to understand your English, though.

digitalstuff74
Community Member

Great work! This has been very insightful.

All the best!

Nage Kaushik
Level 1
Level 1

Best work on EVC I have seen till now!!!! You made understanding EVC easy.

omkarsurve19191
Community Member

Great document. very precise. I literally have no words. I was struggling in l2vpn and had no hopes even after reading device manual, but this document cleared every tiny doubts I had. A big THANK YOU.

Fabrizio Chessa
Level 1
Level 1

That's a very good document!!!

Thanks!!

sarahr202
Level 5
Level 5

Great article, please do think about writing a book!!

I have one question:

 

From your example above:

 

Finally, we can take multiple EVCs, and send all of them over an MPLS Pseudowire

interface g0/2

  service instance 1 ethernet

    encapsulation dot1q 18

    rewrite ingres tag pop 1 symmetric

    bridge-domain 44 split-horizon

  service instance 2 ethernet

    encapsulation dot1q 66

    rewrite ingress tag pop 1 symmetric

    bridge-domain 44 split-horizon

!

interface Vlan44

  xconnect 192.168.1.1 55 encapsulation mpls

 

When sending frame over PW to rempte PE 192.168.1.1, will the sending PE impose vlan tag 44 ?

 

Thanks and have a nice evening!!

 

 

ingrafaeltorres
Level 1
Level 1

Very nice Doc! Very Hepfull! 

manvsing
Cisco Employee
Cisco Employee

great doc


@plumbis wrote:

 

The Problem

On traditional switches whenever we have a trunk interface we use the VLAN tag to demultiplex the VLANs. The switch needs to determine which MAC Address table to look in for a forwarding decision. To do this we require the switch to do two things:

1.) Have the VLAN configured globally

2.) Perform MAC learning in this VLAN

The challenge with this is that it requires us to use finite resources, perhaps without reason. Since the 802.1q VLAN tag is only 12-bits wide we can only configure a maximum of 4096 VLANs. Furthermore switches have a finite amount of CAM space for MAC Learning limiting the number of hosts we can support. 

In modern provider and cloud environments there is a need to scale beyond these limitations.

Enter EVCs

Ethernet Virtual Circuits (EVCs) allow us to leverage existing 802.1q VLAN tags in a brand new way. Traditionally the VLAN tag defined both classification (which VLAN) and forwarding (which CAM table to do a MAC lookup in). Now, with EVCs we can separate these concepts; the VLAN tag is used for classification and the Service Instance defines the forwarding action. For example we could allocate VLAN 10 to different customers on every switchport and forward each customer's traffic across different MPLS Pseudowires, but never actually configure VLAN 10 globally!

How It Works (Ingress)

As an 802.1q tagged frame enters an interface that has been configured with an EVC we will determine which EVC it is classified into based on the tags on the frame. Within the EVC we define what action we wish to do with that frame. Let's take a look at a sample EVC configuration.

interface GigabitEthernet0/2

switchport trunk allowed vlan none 

switchport mode trunk

service instance 6 ethernet

  encapsulation dot1q 10

  rewrite ingress tag pop 1 symmetric

  xconnect 192.168.1.1 33 encapsulation mpls

Now, let's break down each piece of this configuration. 

switchport trunk allowed vlan none 

switchport mode trunk

These lines tell the interface that we need to process 802.1q tags as a trunk interface should, however we will not actually pass any VLANs through this interface. VLAN tags received on Service Instance interfaces have no direct relationship to VLANs configured on the switch. Furthermore Service Instance interfaces do not do any MAC learning (except through a bridge-domain VLAN interface, which is discussed later). Because of this we do not want to allow any globally configured VLANs across this trunk interface. We only need to enable VLAN tag processing and let the Service Instance figure out what to do with the frame.

service instance 6 ethernet

This defines the service instance. The number is arbitrary; it has nothing to do with the VLANs that will be processed by this particular Service Instance The "ethernet" keyword is always used.

encapsulation dot1q 10

This is how we map an incoming tag to a service instance. If VLAN tag 10 is received on this interface it will be put into service instance 6.

rewrite ingress tag pop 1 symmetric

Since the incoming tag no longer has any inherent meaning beyond this specific interface we need a way to discard that tag before forwarding the frame on. The rewrite ingress command does just that. In this case we will remove exactly 1 tag, This command is optional and there are a number options that can be done beyond simply removing the tag including, VLAN translation and imposing additional tags. We will discuss some of these options and the "symmetric" keyword a little later.

Finally, what is our forwarding action with that frame?

xconnect 192.168.1.1 33 encapsulation mpls

This tells us that the frame should be sent across the L2VPN MPLS cloud. Since before this we configure the rewrite ingress tag pop 1 symmetric command we will send a frame with no VLAN Tags across the MPLS pseudowire. There are a multiple possible forwarding actions including routing and local switching (connect)

How It Works (Egress)

As a frame is received it will be classified to go out the Service Instance interface based on how it arrived on this switch in the first place. For example if the frame is received across MPLS pseudowire 33, we automatically know it is part of service instance 6

Continuing to work bottom up in the configuration we come to the symmetric part of rewrite ingress tag pop 1 symmetric. Since we popped 1 tag ingress, to be symmetric we need to push 1 tag egress. The symmetric keyword will always be used with the rewrite command.

We determine which tag to impose based on the encapsulation dot1q 10 command. As the frame egresses the interface impose VLAN Tag of 10. Keep in mind that the access layer device that is sending us tagged frames is most likely a traditional Layer-2 switch and needs the tag it sends to be the same tag it receives for proper classification.

 

Flow Direction.png

 

Step-by-Step Example

Here's a sample topology, with two access switches processing different VLANs. The service instance configurations are on PE Blue and PE Purple.

 

Example Configuration.png

So a few things to note here:

  • The access layer switches are sending and expecting different VLAN tags.
  • The service instance numbers are arbitrary
  • The VLAN tag will be popped before being sent into the MPLS cloud

Frame Example 1.png

A client connect to the access port will send an untagged frame. This frame will be have VLAN tag 10 added to it by the access layer switch and sent to the PE with the service instance configuration.

The Blue PE will see VLAN tag 10 and place it into service instance 9. Since we have configured the 'rewrite ingress pop 1 symmetric' command, we will pop the first tag before applying an MPLS label and forwarding into the MPLS cloud.

Frame Example 2.png

 

As the labeled packet leaves the MPLS cloud we place the untagged frame into PE Red's service instance 18, based on the "xconnect" command. From there since the "rewrite ingress pop 1 symmetric" command is configured and this is an egress frame we know we need to impose one. The tag imposed is based on the "encapsulation dot1q" configuration, so in this case, VLAN tag 11 is imposed on the frame before sending back out to the access layer switch.

EVC Options

Flexible Matching

One of the things that make EVCs so powerful is their flexible matching criteria. EVCs allow us to classify inbound frames in a highly flexible manner based on 1 or more VLAN tags or CoS values. Here are some examples

 

ConfigurationEffect

encapsulation dot1q 10 Match the single VLAN tag 10

encapsulation dot1q 25 second-dot1q 13

Match first VLAN tag 25 and second tag 13
encapsulation dot1q any second-dot1q 22 Match any double tagged frame with a second tag of 22

encapsulation dot1q 16 cos 4

Match a single tag 16 when it has CoS value 4
encapsulation dot1q untagged Match the native (untagged) VLAN
encapsulation dot1q default The catch all class for all traffic not previously classified

 

The options here are not exhaustive but just some examples. The other thing to remember about tag matching is that we follow a longest match criteria.

 

Rewrite Options

Along with a number of flexible matching options we have numerous tag rewrite options.

 

ConfigurationEffect

rewrite ingress tag pop 1 symmetric remove the top 802.1q tag
rewrite ingress tag pop 2 symmetric remove the top two 802.1q tags

rewrite ingress tag translate 1-to-1 dot1q  28 symmetric

remove the top tag and replace it with 28

rewrite ingress tag translate 2-to-2 dot1 22 second-dot1q 23

remove the top two tags and replace them with 22 and 23 (23 will be the inner tag)
rewrite ingress tag push dot1q 56 second-dot1q 55 push two new tags on top of the existing frame. The top tag will be 56; inner tag of 55

 

Forwarding Options

An EVC can be attached to an MPLS xconnect and we can send the traffic across an MPLS cloud.

For more flexibility EVCs introduce the concept of the Bridge Domain. A Bridge Domain is what is traditionally thought of as a Layer 3 SVI. Unlike the VLAN tags that are being processed by the configured EVCs bridge-domains do require the VLAN to be configured globally on the device and use platform wide resources. 

Here is an example of an interface configured with a bridge-domain: 

interface g0/2

  service instance 1 ethernet

    encapsulation dot1q 18

    rewrite ingres tag pop 1 symmetric

    bridge-domain 44

!

interface Vlan44

  ip address 192.168.1.1 255.255.255.0

 

The packet, without VLAN tags, will be passed to the VLAN44 interface for normal routing to occur.

We can also tie multiple service instances to the same bridge-domain to make forwarding tagged traffic highly flexible.

 

interface g0/2

  service instance 1 ethernet

    encapsulation dot1q 18

    rewrite ingres tag pop 1 symmetric

    bridge-domain 44

  service instance 2 ethernet

    encapsulation dot1q 66

    rewrite ingress tag pop 1 symmetric

   bridge-domain 44

!

interface Vlan44

  ip address 192.168.1.1 255.255.255.0

 

This configuration will allow either the service instances to speak between one another or out to other routed subnets. More broadly, Vlan 44 will bridge together the two service instances and forward frames based on MAC learning. As a frame enters Service Instance 1, the VLAN tag will be removed, the frame will be passed to Vlan 44 where the destination MAC will be looked up. If the destination is out the other Service Instance, the frame will be placed on Service Instance 2 and a new VLAN tag will be added. 

Bridge domains also allow for the configuration of a "split-horizon" (bridge-domain 44 split-horizon) to prevent inter-EVC communication, only allowing for routing outside of the bridge domain.

Finally, we can take multiple EVCs, and send all of them over an MPLS Pseudowire

interface g0/2

  service instance 1 ethernet

    encapsulation dot1q 18

    rewrite ingres tag pop 1 symmetric

    bridge-domain 44 split-horizon

  service instance 2 ethernet

    encapsulation dot1q 66

    rewrite ingress tag pop 1 symmetric

    bridge-domain 44 split-horizon

!

interface Vlan44

  xconnect 192.168.1.1 55 encapsulation mpls

 

This will allow us to take two different VLANs and send them to the same MPLS endpoint, removing the VLAN tags in the process. In this scenario we will learn MAC address on both service instances and send them both over the single xconnect, but we prevent traffic on service instance 2 from being sent to service instance 1.

Final Thoughts

Since the way EVCs work is so different from traditional switching not all switching platforms are capable of doing the EVC frame manipulation independently of the forwarding action. Newer platforms like the me3600x or me3800x were designed from the ground up with this kind of capability in mind. The 7600 platform requires newer Ethernet Services (ES) modules to do the additional work that the Supervisor and DFC forwarding engines are unable to do. This guide is also not an exhaustive list of supported platforms or configurations, but merely to demonstrate some deployment options and how traffic forwarding operates in these new EVC environments.


 

superlikf
Level 1
Level 1

great!

mac_mac_net83
Level 1
Level 1

we have been using evc's on our network , particularly as a replacement for legacy dot1q tunneling. I have difficulty understanding it until I read this blog. Thank you for this. May I ask if you have issues when enabling l2protocols such as cdp/stp when using service instances?

hoses
Cisco Employee
Cisco Employee

Really easy to read and understand, Very good summary. Thank you.

Your explanation is clean and clear even now, Thanks a lot.

sim_bambrah
Level 1
Level 1

This helped me fix my issue.  Thanks for the knowledge

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco