Contexts in User Tenants Sharing Common Tenant L3Out

[p-smallwood]

Most guidance I read says that an EPG in a user tenant must use a context (ie VRF) in the tenant where the L3Out is located
However I read in Cisco documentation (link below) that says:

"EPG A and EPG B are in different tenants. EPG A and EPG B could use the same bridge domain and context, but they are not required to do so. EPG A and EPG B are in different bridge domains and different contexts but still share the same l3extInstP EPG"

This seems to offer the useful possibility of EPGs being able to participate in the local routing within their user tenant but at the same time all taking advantage of route leaking to/from a VRF in the common tenant that contains a shared L3Out.

Link to Cisco Documentation

Has anyone successfully tried this and if so what configuration steps were followed?

Kind Regards,

Peter

[Tomas de Leon]

Shared L3out feature is supported in ACI firmware version 1.2(1i) or Later.

Yes, I have configured and tested this use case scenario. There are couple of ways for doing this. In my configurations, I tend to configure the different SHARED services in the Tenant COMMON. This is not necessary but for this use case scenario, I configure the SHARED L3OUT in the Tenant COMMON.

Note: this response assumes that you know how to configure an External Routed Network and all of the Routing Protocol configurations necessary to peer to external Routing Gateways.
The External Routed Network L3 Out is configured and exchanging routes with external gateways.

This use case scenario I will use 3 Tenants: Tenant-Common, Tenant-Black, and Tenant-White.

In the tenant Common
- Create a VRF (common-v1)
- Create an External Routed Network (common-l3-ospf)
- Create a Global Contract (l3out-contract-global)
- Add a Subnet with Scope on the External Routed Network of (common-l3-ospf)
- Provide & Consume (l3out-contract-global) on the External Routed Network of (common-l3-ospf)

ie.
Create an External Routed Network in Tenant COMMON or Tenant USER.
On the External Network Instance Profile (External EPG)

Subnets:
0.0.0.0/0
Export Route Control Subnet
External Subnets for the External EPG
Shared Route Control Subnet
Shared Security Import Subnet
Aggregate Export
Aggregate Shared Routes

Provided Contracts:
l3out-contract-global

Consumed Contracts:
l3out-contract-global

** Note the Scope settings are set to: (Export Route Control Subnet, External Subnets for the External EPG, Shared Route Control Subnet, Shared Security Import Subnet, Aggregate Export, and Aggregate Shared Routes)

- Export Global Contract (l3out-contract-global) to (Tenant-Black) and (Tenant-White)

===========================

In the Tenant-Black
- Create a VRF (black-v1)
- Create a BD (black-bd1)
- Associate the BD (black-bd1) to VRF (black-v1)
- Associate the BD (black-bd1) to L3out in COMMON (common-l3-ospf)
- Create Application Profile (black-ap1)
- Create Application EPG (black-epg1)
- Associate VMM Domain (or other Domain). Choose Immediate for deployment.
- Consume Contract Interface of exported COMMON contract (l3out-contract-global)

===========================

In the tenant White
- Create a VRF (white-v1)
- Create a BD (white-bd1)
- Associate the BD (white-bd1) to VRF (white-v1)
- Associate the BD (white-bd1) to L3out in COMMON (common-l3-ospf)
- Create Application Profile (white-ap1)
- Create Application EPG (white-epg1)
- Associate VMM Domain (or other Domain). Choose Immediate for deployment.
- Consume Contract Interface of exported COMMON contract (l3out-contract-global)

===========================

If you then perform the CLI commands on the leaf nodes (associated with the VRFs Black & White) and the external Routed Gateways:

- show ip route vrf tenant-black:black-v1

Note: The routes learned by the Black VRF should be the Black routes and the external routes.

- show ip route vrf tenant-white:white-v1

Note: The routes learned by the White VRF should be the White routes and the external routes.

- show ip route (on the external gateway)

Note: The routes learned by the External gateway should be the Black routes, White routes and the external routes.

I Hope this helps and thank you for using the Cisco Support Community for ACI.

Cheers!

T.

[Leon]

any idea how to make this work between non-common tenants ? 1.2 advised that the shared L3out can be implemented between any tenants, but I failed to see how this is possible if the L3 is in a user tenant. When trying to associate BD to an L3 profile, your only option is "common/<something>"

[Leon]

Never mind, I figured it out

[eduardopozo56]

Hi Leon, can you guide me on how did you accomplish this?

I ran into the same implementation today and havent been able to use a L3 Out frome one Context in another context (I can add it but I get a fault and cant see the subnets externally)

Thanks in Advance

[Leon]

Are the subnets in the same tenant but different context, or different tenant ? 

[eduardopozo56]

Same tenant, different context. Thanks

[Leon]

Well, what you are trying to do isn't exactly what I was attempting or the OP was talking about. I personally had never had a setup like that.

But here is what I think you need to do to make your configuration right. 

Say if you have the following setup

1.1.1.0/24 in BD A, which tied to CTX A

2.2.2.0/24 in BD B, which tied to CTX B

You will need 2 L3out profile ( one for each CTX ).

L3out A for CTX - A

Under the logical interface profile, assuming that you are using VPC, then you need to create SVI, and pick an encap, define side A and side B IP address for each switch in the pair. The path will be vpc-201-202, for example

L3out B for CTX - B

In this L3out profile, the only difference is that you need to use a different encap, but you can keep the same VPC path.

Then on your router, you will need to include both encapsulations on the trunk interface connected to ACI

Once you have all the above setup, you need to make sure of the following

Subnets are set to "external"

In the bridge domain, under "L3 Configurations", you need to associate your L3 profile to the bridge domain, and make sure "unicast routing" is turned on

[eduardopozo56]

Hi Leon, thanks for the answer.

However, what I wanted to accomplish is to avoid using one L3-out per context. 

I think I managed to do it on my lab, by creating a L3-out on a context (either the common or any other context) and then use contracts and import/export security subnets to control the leaking.

By using the common context I can avoid leaking between other contexts and using per-context global contracts I can apply them between the common L3-out and the in-context BD's

Regards

[Leon]

Then it's a single common context that you are using for both BDs ?

If you don't want to advertise a subnet out of ACI, simply put it in private mode.

And if you don't want to leak routes between context, just don't leak it ?

[eduardopozo56]

Different contexts, but you would still need to leak the route outside.

For example you have Context A and Context B. You want a single L3-out for all the contexts (imagine 150 instead of 2) but you still dont want routes from A in B and viceversa

Per-context contracts and a shared context (only for the L3 out) accomplish this. You cant have the subnet as private or dont leak it because you need it outside but you dont want it in other contexts.

I'll write a quick guide on Monday for the client and also paste it here

[Leon]

Hello, today I tried configuring a shared L3out in common tenant, it appears that this step is not needed

- Associate the BD (black-bd1) to L3out in COMMON (common-l3-ospf)

which I don't quite understand. I know for sure that when using L3out for BDs in the same tenant, the above is needed

Also if you are building a contract between common and a user tenant, I don't think there is a need to export the contract because common contract can be shared

[apache_le]

When you leak the external routes into the user tenant, one of the route will be the default route (0.0.0.0) which provide each of the tenant Internet access.  That means Tenant A can follow the default route and get to Tenant B. But without a contract in place, they cannot communicate, correct?  

[Leon]

There is a contract in place, however, I did not need to associate any layer3 profile within the bridge domain

Also that I did not export/import any contract because common contracts can be shared between tenants anyways

[zafarsohail]

Tomas,

I am new to ACI and wondering if you can help me with this.  i have the L3out configured in a separate tenant as i am connecting a single layer 3 connection from firewall to this "City Tenant" via static route.  How can i configure it in a way that the outside user coming in via the firewall interface is able to get to all the other Shared services from other tenants currently shared with Common tenant.  My other tenants are already tied up with the common tenant all i want to do is somehow let the new City tenant share its L3OUT to firewall with common tenant so every one can talk to each other and let the firewall handle the allow and block filtering.