cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
13369
Views
15
Helpful
20
Replies

Shared L3Out in Common?

bcn-jbrooks
Level 1
Level 1

Hi all,

I'm very new to ACI, but learning as much as I can.  We are in the midst of building up an ACI fabric mostly for lab use, running the latest v2.0 ACI code.

I'm trying to create a Shared L3Out (to the Internet) that all tenants will share.  I have a firewall connected to my ISPs that is connected to the ACI over a vPC.  I have not been able to find much into about how to do this.  Most documentation guides assume that I want to use a "routed interface" (or sub-interface), not a vPC.

This would be simple to do in a traditional network - a small transit network between the firewall and an SVI on the switch, but I can't figure out how to do this on ACI.

Do I need an EPG in the Common tenant for the "vlan" between fabric and firewall, then another EPG that gets Shared to the tenants?

Does anyone have a link to a good tutorial or something?

My mind is officially blown -- and that's after taking several Cisco training courses on ACI... 

Much appreciated!

J

1 Accepted Solution

Accepted Solutions

Tomas de Leon
Cisco Employee
Cisco Employee

Shared L3out feature is supported in ACI firmware version 1.2(1i) or Later.

Yes, I have configured and tested this use case scenario. There are couple of ways for doing this. In my configurations, I tend to configure the different SHARED services in the Tenant COMMON. This is not necessary but for this use case scenario, I configure the SHARED L3OUT in the Tenant COMMON.

Note: this response assumes that you know how to configure an External Routed Network and all of the Routing Protocol configurations necessary to peer to external Routing Gateways.
The External Routed Network L3 Out is configured and exchanging routes with external gateways.

This use case scenario I will use 3 Tenants: Tenant-Common, Tenant-Black, and Tenant-White.

In the tenant Common
- Create a VRF (common-v1)
- Create an External Routed Network (common-l3-ospf)
- Create a Global Contract (l3out-contract-global)
- Add a Subnet with Scope on the External Routed Network of (common-l3-ospf)
- Provide & Consume (l3out-contract-global) on the External Routed Network of (common-l3-ospf)

ie.
Create an External Routed Network in Tenant COMMON or Tenant USER.
On the External Network Instance Profile (External EPG)

Subnets:
0.0.0.0/0
Export Route Control Subnet
External Subnets for the External EPG
Shared Route Control Subnet
Shared Security Import Subnet
Aggregate Export
Aggregate Shared Routes

Provided Contracts:
l3out-contract-global

Consumed Contracts:
l3out-contract-global

** Note the Scope settings are set to: (Export Route Control Subnet, External Subnets for the External EPG, Shared Route Control Subnet, Shared Security Import Subnet, Aggregate Export, and Aggregate Shared Routes)

- Export Global Contract (l3out-contract-global) to (Tenant-Black) and (Tenant-White)

===========================

In the Tenant-Black
- Create a VRF (black-v1)
- Create a BD (black-bd1)
- Associate the BD (black-bd1) to VRF (black-v1)
- Associate the BD (black-bd1) to L3out in COMMON (common-l3-ospf)
- Create Application Profile (black-ap1)
- Create Application EPG (black-epg1)
- Associate VMM Domain (or other Domain). Choose Immediate for deployment.
- Consume Contract Interface of exported COMMON contract (l3out-contract-global)

===========================

In the tenant White
- Create a VRF (white-v1)
- Create a BD (white-bd1)
- Associate the BD (white-bd1) to VRF (white-v1)
- Associate the BD (white-bd1) to L3out in COMMON (common-l3-ospf)
- Create Application Profile (white-ap1)
- Create Application EPG (white-epg1)
- Associate VMM Domain (or other Domain). Choose Immediate for deployment.
- Consume Contract Interface of exported COMMON contract (l3out-contract-global)

===========================

If you then perform the CLI commands on the leaf nodes (associated with the VRFs Black & White) and the external Routed Gateways:

- show ip route vrf tenant-black:black-v1

Note: The routes learned by the Black VRF should be the Black routes and the external routes.

- show ip route vrf tenant-white:white-v1

Note: The routes learned by the White VRF should be the White routes and the external routes.

- show ip route (on the external gateway)

Note: The routes learned by the External gateway should be the Black routes, White routes and the external routes.


I Hope this helps and thank you for using the Cisco Support Community for ACI.

Cheers!

T.

other links

Chapter: ACI Fabric Layer 3 Outside Connectivity
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/basic-config/b_ACI_Config_Guide/b_ACI_Config_Guide_chapter_0110.html

Learning ACI – Part 9: Layer 3 External Connectivity
https://adamraffe.com/2015/03/29/learning-aci-part-9-layer-3-external-connectivity/

Brazos / v1.2 Features – Shared L3Out
http://www.theaciguy.com/?p=69

View solution in original post

20 Replies 20

Tomas de Leon
Cisco Employee
Cisco Employee

Shared L3out feature is supported in ACI firmware version 1.2(1i) or Later.

Yes, I have configured and tested this use case scenario. There are couple of ways for doing this. In my configurations, I tend to configure the different SHARED services in the Tenant COMMON. This is not necessary but for this use case scenario, I configure the SHARED L3OUT in the Tenant COMMON.

Note: this response assumes that you know how to configure an External Routed Network and all of the Routing Protocol configurations necessary to peer to external Routing Gateways.
The External Routed Network L3 Out is configured and exchanging routes with external gateways.

This use case scenario I will use 3 Tenants: Tenant-Common, Tenant-Black, and Tenant-White.

In the tenant Common
- Create a VRF (common-v1)
- Create an External Routed Network (common-l3-ospf)
- Create a Global Contract (l3out-contract-global)
- Add a Subnet with Scope on the External Routed Network of (common-l3-ospf)
- Provide & Consume (l3out-contract-global) on the External Routed Network of (common-l3-ospf)

ie.
Create an External Routed Network in Tenant COMMON or Tenant USER.
On the External Network Instance Profile (External EPG)

Subnets:
0.0.0.0/0
Export Route Control Subnet
External Subnets for the External EPG
Shared Route Control Subnet
Shared Security Import Subnet
Aggregate Export
Aggregate Shared Routes

Provided Contracts:
l3out-contract-global

Consumed Contracts:
l3out-contract-global

** Note the Scope settings are set to: (Export Route Control Subnet, External Subnets for the External EPG, Shared Route Control Subnet, Shared Security Import Subnet, Aggregate Export, and Aggregate Shared Routes)

- Export Global Contract (l3out-contract-global) to (Tenant-Black) and (Tenant-White)

===========================

In the Tenant-Black
- Create a VRF (black-v1)
- Create a BD (black-bd1)
- Associate the BD (black-bd1) to VRF (black-v1)
- Associate the BD (black-bd1) to L3out in COMMON (common-l3-ospf)
- Create Application Profile (black-ap1)
- Create Application EPG (black-epg1)
- Associate VMM Domain (or other Domain). Choose Immediate for deployment.
- Consume Contract Interface of exported COMMON contract (l3out-contract-global)

===========================

In the tenant White
- Create a VRF (white-v1)
- Create a BD (white-bd1)
- Associate the BD (white-bd1) to VRF (white-v1)
- Associate the BD (white-bd1) to L3out in COMMON (common-l3-ospf)
- Create Application Profile (white-ap1)
- Create Application EPG (white-epg1)
- Associate VMM Domain (or other Domain). Choose Immediate for deployment.
- Consume Contract Interface of exported COMMON contract (l3out-contract-global)

===========================

If you then perform the CLI commands on the leaf nodes (associated with the VRFs Black & White) and the external Routed Gateways:

- show ip route vrf tenant-black:black-v1

Note: The routes learned by the Black VRF should be the Black routes and the external routes.

- show ip route vrf tenant-white:white-v1

Note: The routes learned by the White VRF should be the White routes and the external routes.

- show ip route (on the external gateway)

Note: The routes learned by the External gateway should be the Black routes, White routes and the external routes.


I Hope this helps and thank you for using the Cisco Support Community for ACI.

Cheers!

T.

other links

Chapter: ACI Fabric Layer 3 Outside Connectivity
http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/basic-config/b_ACI_Config_Guide/b_ACI_Config_Guide_chapter_0110.html

Learning ACI – Part 9: Layer 3 External Connectivity
https://adamraffe.com/2015/03/29/learning-aci-part-9-layer-3-external-connectivity/

Brazos / v1.2 Features – Shared L3Out
http://www.theaciguy.com/?p=69

Thank you!  I'm getting closer (I think).

I'm confused about this line:

- Provide & Consume (l3out-contract-global) on the External Routed Network of (common-l3-ospf)

I don't see anywhere in the External Routed Network where I can provide/consume contracts.  Do you mean to provide and consume the global contract on an EPG in tenant COMMON?

Sample Screen shot:

Thanks for posting the pic!  i just found it.

my l3out in common is now shared with a user tenant!  woot!

question about exporting the contract to the user tenant... i'm going to be sharing this l3out with hundreds of tenants... it's a bit tedious doing the export to tenant, then import the interface for each, but I'm looking at automating the tenant set up... 

anyway, thanks for the help gents!

The example that I provided use Export & Import for explanantion of sharing contracts between Tenants.  The exception to this rule is contracts defined with with scope of GLOBAL only in the TENANT COMMON; then you can just Provide this contract in the TENANT White & Black EPG.  You do not have export and import.

Also, the best way to replicate building out Tenants is to use the REST API and using POSTMAN (or other REST API app) to post configurations.

Cheers!

T.

Thanks for all your help Tomas!

I'm confused about your last post directly above... I'm experimenting with creating a tenant and associating it to the L3Out.  In the post above, you mention that "you can just Provide this contract in the TENANT [..] EPG."  Why is the tenant Providing the contract in this case and not consuming it?  

Thanks!

Joel

take a look at my example and comments on this post about using the rest api and postman:

https://supportforums.cisco.com/discussion/13100381/copy-apic-config-new-apic

Thanks for all the help guys!

I was curious about this part:

- Associate the BD (black-bd1) to L3out in COMMON (common-l3-ospf)

On my USER tenant Bridge Domain, on the L3 Configurations tab, there's two places to add an L3Out... they are:

Associated L3 Outs

L3 Out for Route Profile

do I need to add it in both places, or just in the list box for Associated L3 Outs?

Thanks!

J

"Associated L3 Outs" links the BDs to the configured L3 Outs for route advertisement of Subnet(s).

The Route Profile is if you configure a "Custom" route profile and you want it associated to this particular BD.  Typically most people just a associate the BDs.

It just depends on what you are trying to do...

Cheers!


T.

Hy Tomas!

Should this Shared L3Out Config be also possible when I configure the "Common" part in a User Tenant? I think I have read that this should be possible now (ACI Version 2.0).

I have tried to configure that, but it does not work, I cannot associate the L3-out in the consumer tenants?

Thanks!

you need to create a provider contract in the tenant your L3out is in ( tenant A), make the scope global. Then export that contract to tenant B. Once exported, you can consume the contract in either the EPG or VRF in tenant B depends on your requirement.

consuming the contract in VRF, makes the L3out available to ALL the subnets associated with that VRF

consuming the contract in EGP makes the L3out only available to that EPG.

What about if you have routes in a L3Out on a user tenant and want to export it to to a shared L3Out in common (transit with a shared) ? seems even by making the scope of a contract global the external shared router does not receive the internal L3out route.

make sure that you are on version 1.2(1i) - I think - or above.

If you need to export a L3out from user tenant to common, then you can simply create a contract in common first. Then associate the contract with your L3out EPG in your user tenant, as provider.

In your common tenant VRF or specific EPG, assign the contract as consumer. I have never personally done it this way, if it doesn't work, then try this

Create the contract in user tenant, assign it to the L3out profile EPG as provider.

Export the contract to common tenant, and then assign it to the EPG or entire VRF as consumer contract interface

Leon,

The user tenant L3Out is a static route so even by exporting as provider a contract to common shared L3out or using a contract living in common the route does not appear in common:default vrf table. The User L3out is set as shared as well.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Save 25% on Day-2 Operations Add-On License