One Bridge Domain One Subnet Multiple EPG

[rachmatadhie]

hi all,

i've deployed aci for one of my customer and it is using network centric approach which is one subnet one BD one EPG and its worked like desired. but there is a need that require to configure aci to have one subnet one BD multiple EPG and multiple VLAN. for example i have one subnet called subnet A that associated with one BD called BD A . that BD contain one existing epg called EPG A and this EPG have static port trunking vlan x interface X . then i created a new EPG called EPG B associated to BD A and this EPG have static port trunking VLAN Y interface X. this two epg is in the same physical domain , AEP and vlan pool. when i assigned static interface X with trunking vlan Y, the existing endpoint contained in EPG A is cannot communicate to every where including the gateway that reside in aci leaf.

is there anyone who can give me an advice about this issue??

thank you,

Adhie

[Manuel Velasco]

Where is the gateway for EPG-A and EPG-B?

What happens when you only add a the static binding for EPG-B?

Have you ran any span captures on any endpoint in EPG-A after you add the static binding for EPG-B?  One possibly is that there might be a lot of broadcast traffic from endpoints in EPG-B that is causing endpoints in EPG-A to have communication issues, as you know in ACI the BD represents the broadcast domain. 

[rachmatadhie]

mr. velasco,

the gateway is in leaf switch for both EPG.
when i bind the static port only to one of those epg it is work normally.

i haven't run any span capture on any endpoint and i think it is not broadcast issue from my endpoint because when i bind the static port only to one of those epg it is work normally.

thankyou

[Manuel Velasco]

Are both of these EPGs using the same GW or different on the same BD?

when you add the second static binding for EPG-B, can are the endpoints in EPG-A able to reach their GW?

Are there any faults under either EPG?

If endpoints in EPG-A are not able to reach their GW, have you checked the ARP table on these endpoints?

Have you checked the status of this  interface or PC after you add the static binding? 

[rachmatadhie]

This two epgs using the same gateway in the same BD

After i added the static interface for epg b the endpoint contained in epg a cannot communicate to anywhere including its gateway

There is no fault in both this two epgs , both health score is 100

I can’t remote neither direct access to my devices in epg a, i can’t remote because i lost my connectivity to them, i cant direct to it because this deployment is for DR and the only way to access it is via remote

the interface that used for static path for this two epg is up nothing down

[Beau Poehls]

What sort of device is connected to interface X? Switch or Server? What mode are you using for the Static EPG binding (Trunk, Access - 802.1p, or Access - Untagged)? Can you confirm there are no faults under each EPG? If not, can you post what faults you're seeing?

[rachmatadhie]

It is actually esxi server and using trunk interface. There is no fault in both epgs

[stears]

you need to configure different MAC Addresses. Otherwise the same MAC is going in opposite directions, and creating issues in the mac forwarding table.

i have tested: 

example: on ACI - Single BD: (BD_10), with 2 EPGs (EPG_10), (EPG_11) with 2 VLANs (Vlan10), (Vlan11)  

BD_10 <> EPG_10 <> Vlan_10 <> VPC_Intf_10 Static Port 

BD_10 <> EPG_11 <> Vlan_11 <> VPC_Intf_10 Static Port 

I had "as Endpoints" the Vlan Interfaces on another Nexus3k Switch

on NX3K (attached to ACI VPC_Intf_10), configuration on NX3K:

VRF_10, Interface Vlan 10, ip addr 10.0.0.10/24 (member in VRF_10) 

VRF_11, Interface Vlan 11, ip addr 10.0.0.11/24 (member in VRF_11)

1st i could Not ping between the 10.0.0.10 and 10.0.0.11. Expected to go up to ACI and back to the N3K from one Vlan into the other.

But when i configured different MAC-address on the interfaces Vlan 10 and Vlan 11 on the NX3K >> then the Ping worked.

the explanation with the "MAC Address confusion" is on the Link below. But the solution offered there is "Flood in Encapsulation", which didn't fit the wanted goal. The other workaround-solution was to configure different MAC Addresses on the Vlan interfaces.

on NX3K:

VRF_10, Interface Vlan 10, ip addr 10.0.0.10/24 (member in VRF_10), mac-address 10:00:00:10:10:10

VRF_11, Interface Vlan 11, ip addr 10.0.0.11/24 (member in VRF_11), mac-address 10:00:00:11:11:11

https://www.cisco.com/c/en/us/td/docs/dcn/aci/apic/6x/l2-configuration/cisco-apic-layer-2-networking-configuration-guide-60x/bridging-60x.html#id_59068