Internal EPGs reachable from L3out

[Antonio Macia]

Hi,

I'm trying to understand why I can reach endpoints in EPGs from the outside without providing any contract. These EPGs consume the L3out contract from the common tenant that allows any outgoing traffic. However, the EPGs don't have any provided contract being consumed by the L3out subnet that would allow the incoming traffic. Actually, they are not providing any contract at all.

Both, the common and EPGs' tenant share the same VRF (common) and enforcement at the VRF level is enabled. Although they initially belonged to the preferred group, I tried to exclude them out of the preferred group for testing but it didn't make any difference.

Any idea?

Thanks.

[ralphcarter]

Check to see if the VRF is set to "unenforced".

[Antonio Macia]

Hi ralpshcarter,

As I explained above, the VRF is set to enforce.

[Claudia de Luna]

Hi @Antonio Macia,

Can you clarify "how" you can reach the endpoints (ICMP I'm assuming..)?  Some infrastructure traffic like DHCP is permitted.  Also, is the default contract associated with any of the EPGs (in either direction)?   

A few other suggestions:

- Review your contract scopes

- Is the "Stateful" option checked?  That's a handy little feature that just requires that the ACK bit be set (obviously wont' work for non-TCP traffic).

You have already checked the other two places I'd look (VRF enforcement and unintended inheritance via preferred group).

[Antonio Macia]

Hi Claudia,

Thanks for the response.

Apart from ICMP, SSH traffic also works. I created a custom contract with a custom subject and filter with the purpose of allow all the outgoing traffic. Because the filter applies to any protocol, the "stateful" flag is set to false.

I moved between global and VRF scope and the incoming traffic still works. Only stopped working if I changed to tenant (as expected), but this is not desired.

[Remi-Astruc]

Hi @Antonio Macia ,

If you have enabled the Subject setting "Apply Both Directions" (which is default), and the Filter with Unspecified protocol, the traffic is also allowed from Provider to Consumer on Any protocol...

Try to restrict your Filter with Destination TCP SSH, and you will not be able anymore to SSH the EPG from L3Out (only SSH answers with SSH as source port will be allowed).

[Antonio Macia]

Hi Remi,

So a contract allowing any protocol grants access from the provider to the consumer. What if I just need to block any incoming traffic to the fabric while allowing all the outgoing? Not for a specific protocol in particular. How could I achieve that? Should I use a L4/L7 device (firewall) or can I leverage contracts to do this?

I tried disabling the "Apply both directions" setting and allowing the traffic in just one direction but it blocks all the traffic, I suppose because of the stateless condition of the contracts.

[RedNectar]

Hi @Antonio Macia ,

Firstly read this discussion to help you understand the Apply in both directions https://community.cisco.com/t5/application-centric/aci-contract/td-p/3855931 at least the first part anyway.

When I read "These EPGs consume the L3out contract from the common tenant that allows any outgoing traffic" I think you may have misunderstood something. There is no intrinsic contract in the common tenant, so if you have created one, you'd better tell us what it is, but as far as I know the only possible way to create a contract that allows ONLY outgoing traffic would be TCP specific, and would have to have the Stateful flag set.

So the question is - what exactly is the contract?  I think this would be a good place to start rather than speculating about ICMP, SSH or any other protocol.

[Remi-Astruc]

Hi @Antonio Macia ,

You should see the contracts similar to traditional ACLs. They are stateless.

There is no way to exhaustively implement what you require with contracts (Any IP outgoing, only answers incoming), but you can do some tricks to tend to it:

- Disable "Apply both directions" in your IP Any contract subject, so that any UDP/TCP is allowed to egress.

- Create a second contract to allow TCP answers, with Disabled "Apply both directions" and a filter TCP Any with TCP Session Rule "Established". Provide that contract from EPG and consume by L3Out.

But you cannot do the same for UDP traffic.

"Stateful" flag may not be applicable here. That feature is only relevant for AVS/AVE VMM Switches.

[Antonio Macia]

Hi @Remi-Astruc ,

It seems that I misunderstood the concept.

Having a contract provided by the L3out and consumed by a different EPG injects the default-route on the consumer EPG so it has outside connectivity.

Then we have the security policy part between the EPG and the L3out, which should be complemented with a firewall, whether it be in the form of L4/L7 service graph or service insertion, am I right?

Thanks.

[Remi-Astruc]

Hi @Antonio Macia ,

That's not that way.

The L3Out route is advertised from the Border Leaf to the other Leaves of the VRF, but that's a bit off-topic.

Back to contracts, a contract is a security rule (ACL) between EPGs, which controls if the traffic is allowed or dropped. In some cases, the contract can be added a Service Graph which inserts L4L7 device on the path (FW, LB), but that is still a contract.