Pure L2 VLAN on Cisco ACI with Gateway outside the fabric

[CollisionDomain]

So we have two sites, one with ACI and one without. In the site where there is no ACI, for a project PoC, we needed to deploy a new VLAN on the core switches and extending it to other access switches but no SVI for the VLAN. Basically, L2 VLAN only. The gateway for the VLAN will be on the firewall. 

So for eg, we created VLAN 100 on the core, assigned two access ports to the vlan, one port goes towards another access switch which extends the VLAN 100 to other switches and other access port which connects to the firewall and the firewall has the gateway for VLAN 100 configured.

How can I achieve the same on Cisco ACI? It's a fairly straightforward ask, but I want to ensure I am doing it the right way. I want to create an L2 VLAN on the ACI fabric whose gateway is outside the fabric. How can I do this? My knowledge on ACI is on-off, due to lack of practice. Please help.

[Remi-Astruc · Accepted solution]

Hi @CollisionDomain ,

Don't know from where you start, but assuming your Fabric is already set with Domains/AEP/Switch/Interface Profiles/Tenant/VRF, you need to create a Bridge Domain with all L2UU/Multi-Destination/ARP set to "Flood" and no Unicast Routing, create an EPG attached to that BD, and assign the server and FW ports to that EPG using EPG Static Port Binding (Port type, Port-encap Vlan 100, Mode trunk/access). That's basically it.

Keep in mind that using ACI as L2 only prevents you from many benefits the Fabric can bring.

Regards

[CollisionDomain]

I should have mentioned about my existing fabric in the post. My bad. Yes, all of that is already setup. Your answer was helpful!

[Ali Aghababaei]

Hello @CollisionDomain 

To configure a Layer 2 VLAN on Cisco ACI with the gateway outside the fabric, you can follow these steps to ensure the VLAN traffic is handled correctly while keeping the gateway on an external firewall:

### Steps to Configure a Pure L2 VLAN on Cisco ACI:

1. **Create a Bridge Domain (BD)**:
- Navigate to **Tenant > Networking > Bridge Domains**.
- Create a new Bridge Domain for your VLAN. Ensure you disable "Unicast Routing" because the gateway is outside the ACI fabric.
- Enable "ARP Flooding" to allow ARP requests to propagate through the network.

2. **Create an Endpoint Group (EPG)**:
- Navigate to **Tenant > Application Profiles > [Your Application Profile] > EPGs**.
- Create an EPG and associate it with the Bridge Domain created earlier.
- This EPG will be used to handle the traffic for VLAN 100.

3. **Static Binding of Ports**:
- Navigate to the EPG and select **Static Ports**.
- Bind the physical ports to the EPG that are connected to your core switch and firewall.
- Configure the ports to operate in "Trunk" mode if the traffic will be tagged with VLAN 100, or "Access" mode if it is untagged.

4. **Policy Configuration**:
- Ensure that the correct policies are applied to these ports, such as interface policies and VLAN pools.
- Make sure these policies align with your security and network requirements.

5. **External Network Configuration**:
- Navigate to **Tenant > Networking > External Networks > L2Out**.
- Create an L2Out connection that represents the external network (the network outside the ACI fabric).
- Select the Bridge Domain you created earlier and configure the appropriate VLAN (e.g., VLAN 100).

6. **Associate VLAN with L2Out**:
- Configure the L2Out to map the VLAN 100 to the appropriate physical ports.
- This step ensures that VLAN 100 traffic can leave the ACI fabric and reach the external firewall where the gateway is configured.

By following these steps, you can extend a Layer 2 VLAN in Cisco ACI to an external gateway, enabling seamless integration with non-ACI environments and external firewalls.

[RedNectar · Accepted solution]

Hi @Ali Aghababaei & @CollisionDomain ,

I really thought Ali was saying the same thing as @Remi1 (who has described the bones of what you need to do) but with more detail.

Until I got to Step 5 - @CollisionDomain please don't do Step 5 or 6 - i.e DO NOT CONFIGURE ANY L2Outs

And further, I'd suggest you start do Step 4 from Ali's list before Step 1

To avoid confusion with Steps 5 & 6, let me spell it out:

Note: in the menu sequences below >+ means right-click and the double >> indicates the transition from the top level menus to the side navigation pane, and >| indicates the transition from the side navigation pane to the work pane

Step 1: Access Policy Configuration (i.e. create an Access Policy Chain) = Ali's step 4

1. Create a VLAN Pool [Fabric > Access Policies >> Pools > VLAN >+ Create VLAN Pool]
   1. For illustrative purposes, I'll call this Pool MappedVLANs_VLAN.Pool
   2. Make it a Dynamic VLAN Pool
   3. Add a Static block of VLANs that includes VLAN 100
2. Create a Physical Domain [Fabric > Access Policies >> Physical and External Domains > Physical Domains >+ Create Physical Domain]
   1. For illustrative purposes, I'll call this Domain MappedVLANs_PhysDom
   2. Link the Physical Domain to the MappedVLANs_VLAN.Pool
3. Create an AAEP [Fabric > Access Policies >>Policies > Global > Attachable Access Entity Profiles >+ Create Attachable Access Entity Profile] 
   1. For illustrative purposes, I'll call this AAEP L2Links_AAEP
   2. Link the AAEP to the MappedVLANs_PhysDom
4. Create an Interface Policy Group [Fabric > Access Policies >> Interfaces > Leaf Interfaces > Policy Groups > Leaf Access Port >+ Create Leaf Access Port Policy Group OR, if the links are to be VPCs, Fabric > Access Policies >> Interfaces > Leaf Interfaces > Policy Groups > VPC Interface >+ Create Leaf VPC Interface Policy Group] 
   1. For illustrative purposes, I'll call the Interface Policy group L2Hosts_APPG (if it is an Access Port Policy Group) or xxxx..yyyy:1:nn_VPCIPG if it is a VPC Interface Policy Group, where xxxx is the leaf ID of the first leaf of the VPC, and yyyy is the 2nd leaf of the VPC and nn is the port number that the VPC connect to)
      1. If more than once VPC is involved, you'll need a different VPCIPG for each VPC
   2. Add whichever policies you want to the Policy Group, such as system-cdp-enabled if you wish etc
      1. If it is a VPC, make sure you include a Port Channel Policy, like system-lacp-active
   3. Most importantly, make sure you link the Policy Group to the L2Links_AAEP
5. Assuming ACI v5.2(7g) or later, navigate to Fabric > Access Policies >> Interface Configuration
   1. Under the Actions menu, choose Configure Interfaces
   2. Choose the Interface(s)
   3. Select either the L2Hosts_APPG or xxxx..yyyy:1:nn_VPCIPG as appropriate
   4. Repeat for all relevant interfaces leading to the L2 Hosts and to the other data centre

Note: If using ACI with an version earlier than V5.2(7g), you'll also need to create Interface Profiles with Interface Selectors that define the actual physical ports, and link those Interface Selectors with the appropriate Interface Policy Groups, AND create Leaf Profiles and link them to the Interface Profiles. There's a tutorial on the Access Policy Chain out there that I wrote in 2015 so still refers to Interface Profiles, Interface Selectors and Leaf Profiles. 

---

Step 2: Tenant Policy Configuration = Ali's Steps 1-3

1. Create a Tenant, [Tenants > Add Tenant] give it a name. No need to create a VRF yet since you are doing L2 Only
2. Create a Bridge Domain (BD) within the Tenant [Tenant >> Networking > Bridge Domains >+ Create Bridge Domain]
   1. For illustrative purposes, I'll call this BD L2_BD
   2. During BD creation, choose Custom for the Forwarding option
      1. Set L2 Unknown Unicast to Flood
      2. Set Multi-Destination Flooding to Flood in BD or Flood in Encapsulation
   3. When configuring L3 Configuration for the BD
      1. Set Unicast Routing to Disabled
3. Create an Application Profile within the Tenant [Tenant >> Application Profiles >+ Create Application Profile]
   1. For illustrative purposes, I'll call this AP L2VLANs_AP
4. Create an EPG within the Application Profile [Tenant >> Application Profiles > L2VLANs_AP >+Create Application EPG]
   1. For illustrative purposes, I'll call this EPG VLAN100_EPG
   2. for the Bridge Domain, link the EPG to the L2_BD
5. Link the EPG to your Physical Domain [Tenant >> Application Profiles > L2VLANs_AP > Application EPGs > VLAN100_EPG > Domains >+ Add Physical Domain Association]
   1. Select the MappedVLANs_PhysDom 
6. Link your EPG to the relevant VLAN + Ports/VPCs
   1. Method #1 Mapping Down. [Tenant >> Application Profiles > L2VLANs_AP > Application EPGs > VLAN100_EPG > Static Ports >+ Deploy Static EPG on PC, VPC, or interface
      1.  Select Port or VPC as appropriate for Path Type
      2. Choose the relevant Node/Port or VPC Interface Policy Group
      3. Set the Port Encap (or ... Micro-seg) to VLAN 100
      4. Leave the encapsulation Mode as Trunk
      5. Repeat for all relevant ports/VPCs
   2. Alternate Method #2 Mapping Up. (Not necessary if Method #1 chosen) [Fabric > Access Policies >>Policies > Global > Attachable Access Entity Profiles > L2Links_AAEP >| Application EPGs section]
      1. Click the +  to add and EPG tot he AAEP
      2. Choose Your Tenant, the L2VLANs_AP Application Profile, the VLAN100_EPG  EPG and enter vlan-100 as the Encap
      3. Job done - no repeating for other interfaces using Method #2 Mapping Up.

---

Step 3 Test

If you have completed the steps as outlined above, your L2 Endpoints will be able to ping their default gateway IP on the firewall.

I repeat - do not touch or configure anything to do with L2Outs. There is nothing a L2Out can do that can't be done with a regular Application EPG, and there is MUCH more that can be done eith an Application EPG than a L2Out.

---

Footnote:

Something bugs me about Ali's answer - how could he have got it so wrong to suggest you create both an EPG AND a L2Out? Was any AI engine at work perhaps??????

[CollisionDomain]

Wow! This was extremely on spot! Very helpful. I will follow these steps and get back to you if I hit a roadblock of some sorts.

[CollisionDomain]

@RedNectar I've a follow up question - How would this setup change if we were to advertise that particular network within the fabric linked with VLAN 100 but the gateway still exists external to the fabric?

For example, in the traditional setup, I will create a static route for the network (VLAN100) pointing towards the firewall and then redistribute that static route into OSPF/BGP as needed. But not sure about ACI

[RedNectar]

Hi @CollisionDomain ,

In the scenario I described for you earlier, ACI is nothing more than a Layer2 transport. Functionally equivalent to an ancient Catalyst L2 switch, so There is no need to configure any static routes on the ACI "L2 switch". In fact, since there is no L3 configured on ACI, there is no way you can configure any L3 routes".

In your case, you'll be relying on the endpoints having the static route to their default gateway already set up - which would be the normal situation, and that's all you need.

[CollisionDomain]

That makes sense. However, and this is just for my knowledge, what would be ideal solution if let's say that behind the ACI fabric, we've a bunch of L3out configured and I would also like to advertise this network (VLAN100) to the L3outs while the gateway for the endpoints still stays on the firewall? 

[CollisionDomain]

Maybe my question was not clear, sorry about that. I've attached below diagram for more clarity. Assuming the setup is as below, VLAN100 related stuff is purely L2 with gateway on the firewall, how would I advertise VLAN100 network to the l3out towards rest of my network? Basically, how can we rest of the network also reach that particular network? This is for a PoC we're doing and is a requirement for setup it up in this way. 

@Wassim Aouadi @RedNectar Appreciate your patience in answering my queries. 

[RedNectar]

HI @CollisionDomain ,

Thanks for the diagram - I started looking at your problem and decided I'd need to draw a diagram to see if what you are trying to do is as described in the diagram. I'll repeat it here for ease of reading

So - the problem is that the L3Out in ACI is part of a VRF.  The L2 component (VLAN 100) is NOT part of any VRF because there is no L3 for VLAN 100.

Now let's think about a packet going from your 192.168.1.10/24 to the "Rest of the NW"

Firstly, it's going to go to the FW.

But what do you expect the FW to do with that packet?  Typically, a FW will NOT send a packet back out the interface which it came in on, so you'd need a 2nd interface on that FW that somehow has access to "Rest of the NW" - and if that access is via a L3Out in ACI, then you'll need to configure that - perhaps using a different VLAN on the same physical interface as VLAN 100. Let's say you configure IP 192.168.2.1 on VLAN 101 on the FW, and in ACI, you configure an BD in a VRF (say VRF1 - assuming the L3Out is configured in VRF1) with an IP of 192.168.2.2 and add an EPG with a static mapping to the same physical interface but on VLAN 101.

On the FW, you'd configure a static route of "Rest of the NW" via 192.168.2.2

And in ACI, you'd configure the BD where 192.168.2.2 lives to "Advertise Externally", and add the L3Out to the "Associated L3 Outs" for the BD.

And of course, there will be the rest of the whole L3Out configuration based on the routing protocol (OSPF/BGP/EIGRP/static routes) that the Router is using.  If you need help with that configuration, then there's lots of help out there for that.

---

Now before I wrap up, I'll mention PBR Policy Based Redirection.

IF you only have certain types of traffic that you want 192.168.1.10 to be able to send to the "Rest of the NW" without traversing the FW, then you could move the 192.168.1.1/24 IP address to ACI, and turn your L2 BD into a L3 BD.

You'd then set up a contract that directs that "certain type of traffic" to the "Rest of the NW" and sends all other traffic to the FW.

But that is a whole new discussion...

---

[CollisionDomain]

Hi @RedNectar 

I've a follow up question on this. How would the config change if I wanted to move the gateway out of ACI fabric to a firewall which is connected via an L3out (via SVI's in logical node profile) and using OSPF. How would I move the gateway behind the firewall in this case? 

[RedNectar]

Hi @CollisionDomain ,

Not quite sure what you want to achieve. Reading your question verbatim it sound like you want a default gateway to be on a different L3 subnet, which of course does not make sense. Maybe a diagram would help.

[AlexI1978]

In Step 2 you also should assiciate phy_domain correct? 

[Wassim Aouadi]

Hello @CollisionDomain ,

I don't see where this could make sense, because the VLAN100's network is "neither external nor totally internal" from a layer-3 perspective, to be eligible for integration with an L3Out.