Multi-Site L2 Stretch and Contracts

[JL-CNO]

We currently have Multi-Site setup. We also have a L2 stretch between sites for just a couple EPG's. The Bridge-Domains are where the gateways are configured. 

Scenario 1: without contracts deployed on the L2 stretched EPG's. Two connections, ISN and L3Out WAN, VZAny deployed in both sites.
Site 1 EPG A not stretched, Site 2 EPG B not stretched
EPG C stretched between sites 1 and 2 using the ISN

Site 1 EPG A sends traffic destined for Site 2 EPG B traffic everything works as expected, traffic flows over the L3Out

EPG C can only comminucate across the ISN within the EPG C and out the L3Out, but not to any EPG's in either site.

Scenario 2: contract deployed on the L2 stretched EPG to be able to communicate with EPG B. Two connections, ISN and L3Out WAN, VZAny deployed in both sites.

Site 1 EPG A not stretched, Site 2 EPG B not stretched
EPG C stretched between sites 1 and 2 using the ISN

Site 1 EPG A sends traffic destined for Site 2 EPG B but traffic fails because a route was installed for EPG C that says any traffic destined for said subnet(EPG C bridge-domain) needs to flow over the ISN, but because there is no contract on between EPG A and EPG B the traffic is denied. 

This poses a problem. What solutions I have found. Deploy contracts to all EPG's in Site 1 to allow communication to EPG's in Site 2, which then is forcing the traffic over the ISN which is not optimal, or we have to move all the subnets to the EPG's, create another VRF, move the stretched EPG's to their own VRF, then route leak between the two. 

Hoping someone may have another solution. Thank you ahead of time.

[AshSe]

It would be better, great, and easy to understand if you diagrammatically present your query. 

[AshSe]

@JL-CNO For a better understanding; I have tried to draw your diagram. I am sure it needs improvement/clarity w.r.t. BD. Feel free to improvise in the attached ppt.

[JL-CNO]

@AshSethis looks good. Besides the BD's for EPG A and B are not streched, only EPG C BD is stretched.

[AshSe]

@JL-CNO  Please confirm now:

[AshSe]

@JL-CNO if above diagrams are correct then I have a question (before we move to the solution) to you:

• Why do you need L3Out to connect EPGs of two different sites? If am not wrong they can  be reached via ISN.

It's cool that you have deployed vzany and you don't need a separate contract between EPG-A & EPG-B

In Scenario-2, Since you have vzany contract, do you still need a separate contract between EPG-B and EPG-C. Did you try communicating between them using vzany?

[JL-CNO]

So they cannot be reached across the ISN unless they are stretched with a contract, which forces traffic over the ISN. Traffic going out the L3Out is sent through a firewall which handles the ACL's. Traffic traversing the L3Out works fine, as long as the EPG isn't stretched.

To your second question, something interesting about this is, traffic coming from EPG-C with the server located in Site 1 to EPG-B goes over the L3Out but it stops after hitting the gateway of EPG-C the second time. I am not sure why it sends it to the gateway the second time after it hits the gateway of EPG-B.