Internal Packet Flow with ACI service insertion/graph with transparent firewall

[tsmarcyes]

Ok, I have read over the documentation multiple times on service insertion and I cant seem to find a good explanation for the internal packet flow with an ACI service insertion with a L2/transparent firewall.

So, specifically, Im looking the solution with the L2 transparent firewall logically sitting in the path between 2 BDs.

Take a simple topology - Host1 in EPG1 in BD1 with IP of 10.1.1.100/24 trying to talk with host2 with IP of 10.2.2.2.200/24 in EPG2 in BD2.  There is a service insertion to direct traffic from EPG1 to L2FIREWALL to EPG2.

So from a packet perspective, how would the traffic internally flow and how would it look from the firewall perspective?

Would it be something like the below?

Host1 --- BD1-AnycastGW(10.1.1.1) --- L2Firewall --- BD2-AnycastGW(10.2.2.2) --- Host2

Would the L2firewall see the packet like below?

srcmac = leaf, srcIP = 10.1.1.100, dstmac = leaf, dstIP = 10.2.2.200

If that is correct, how does this scale with multiple BDs/EPGs that need to traverse over the L2 firewall?  Take 4 hosts in 4 EPGs in 4 different BDs.  Does ACI treat that this as all separate paths like below?

Host1 --- BD1-AnycastGW(10.1.1.1) --- L2Firewall --- BD2-AnycastGW(10.2.2.2) --- Host2

Host1 --- BD1-AnycastGW(10.1.1.1) --- L2Firewall --- BD3-AnycastGW(10.3.3.3) --- Host3

Host1 --- BD1-AnycastGW(10.1.1.1) --- L2Firewall --- BD4-AnycastGW(10.4.4.4) --- Host4

Host2 --- BD2-AnycastGW(10.2.2.2) --- L2Firewall --- BD3-AnycastGW(10.3.3.3) --- Host3

Host2 --- BD2-AnycastGW(10.2.2.2) --- L2Firewall --- BD4-AnycastGW(10.4.4.4) --- Host4

Host3 --- BD3-AnycastGW(10.3.3.3) --- L2Firewall --- BD2-AnycastGW(10.4.4.4) --- Host4

Furthermore, does ACI treat these as all separate logical "connection arms" and thus I would need 12 interfaces (6 on each side) on my firewall?  Or would ACI know to reuse the same logical "connection arm" even if its a different path so I would only need 4 interfaces (one to each BD from the firewall).  For instance, take just the last 2 entries from above:

Host2 --- BD2-AnycastGW(10.2.2.2) --- L2Firewall --- BD4-AnycastGW(10.4.4.4) --- Host4

Host3 --- BD3-AnycastGW(10.3.3.3) --- L2Firewall --- BD2-AnycastGW(10.4.4.4) --- Host4

Would ACI create just 3 logical connections arms to the firewall like below:

1.  BD2-L2Firewall         2.     L2Firewall - BD4

3.  BD3-L2Firewall

Or would ACI create 4 logical connection to the firewall because it perceives the second path to BD4 as separate arm because its initially coming from a different BD?

1.  BD2-L2Firewall         2.    L2Firewall-BD4

3.  BD3-L2Firewall         4.    L2Firewall-BD4

I know this would be more of a firewall question but its definitely closely related.  In the multiple EPG scenario above, on the firewall, would I treat all these interfaces as belonging to the same layer2 domain (ie bridge-group in Cisco ASA, forwarding domain in Fortigate, etc)

[6askorobogatov]

Service  insertion is associated with contact Subject. 

If you have an EPG1 and EPG2 and contact Consumer-Provider. In contract you have Subject (let's say Any to TCP/80).  

Now you create service graph and BPR policies and attach to that subject.  Now any packet that hit the zoning rune with that subject will be redirect by front PBR to L2 FW (or whatever) and return packet will be redirected with back PBR to the same L2 FW. 

Now if implement the same contract between EPG 3 and EPG 4 it will be the same.

If you have another contact and/or subject (let's say UDP/53) you can use the same service graph and PBRs for that subject and it will works same way. 

[tsmarcyes]

Thanks @6askorobogatov .  However, that is more high level and more in relation to ACI with the service insertion and traffic redirect.  Im looking for more of the specifics after configuring the redirect.. the exact packet flow within ACI and how it will reach the firewall, how the packet would look like in the firewall logs, logical connections on the firewall/interfaces on the firewall.  

For instance, you mention it will be the same if we do from EPG3 and EPG4.  That may be true from the ACI process and step standpoint.  But again, from the firewall perspective, would it be the same interfaces on the firewall or different interfaces, different connection legs, different bridge domains, different L2 sources/destinations, etc?

[6askorobogatov]

You are looking at packet flow in context of underlay. There will be vxlan from the source to  FW and from FW to the destination. 

Same thing for other source-FW-destination. All flows will use to the same interface(s) on FW, if the same service graph is used. 

L3 device could be deployed "one arm" for in and out on the same service BD. L1/L2 should have in /out on the deferent leafs. 

FW will see source / destination exactly like ACI sees  endpoints .