PBR service insertion for ASA in L2/transparent mode

Micheline Murphy · ‎07-10-2020

Hello all--I'm working on a transparent mode service insertion. I'm using an ASA 5510 running 9.17. From the PBR white paper https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-739971.html#Optionalconfigurations, I understand that when the PBR elements of the service insertion are triggered, the destination MAC is re-written so that it is a MAC purported to be on the "other" side of the ASA. In turn, the ASA needs to have some static MAC addresses so that it just directs the traffic thru itself.

My topo is:

Leaf 101 e1/1 --- e0/1 (inside) ASA e0/0 (outside) --- Leaf 102 e1/1

Unfortunately, I'm getting lost trying to configure the MAC forwarding. I configured two PBR Policies and per the white paper, left the MAC and IP blank. I then went into the APIC CLI and grabbed the MACs that were statically configured to use to statically configure MAC routes in the ASA.

cnat-apic-02# fab 101 show mac address-table
----------------------------------------------------------------
Node 101 (cnat-nexus9348gc-fxp-1)
----------------------------------------------------------------
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 20 0050.56b7.68c4 dynamic - F F eth1/50
* 21 0050.56b7.28bd dynamic - F F eth1/50
* 30 0257.b7b5.1e9b static - F F eth1/1

cnat-apic-02# fab 102 show mac address-t
----------------------------------------------------------------
Node 102 (cnat-nexus9348gc-fxp-2)
----------------------------------------------------------------
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False
VLAN MAC Address Type age Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 20 0291.b92a.a30a static - F F eth1/1
* 4 5c71.0d53.4ac8 dynamic - F F eth1/52

ciscoasa# show mac-address-table
interface mac address type Age(min) bridge-group
-----------------------------------------------------------------------------------
outside 0257.b7b5.1e9b static 1
inside 0291.b92a.a30a static 1

The VMs in each EPG can ping their gateway address, and I checked the zoning-rule table. I've got a PBR redirect programmed in there between my consumer and provider EPG. The ASA isn't configured with any filtering at this point, just (theoretically) passing traffic from the inside interface to the outside interface. I've added all the ASA config listed in the PBR white paper except the IP SLA (since I only have one ASA).

That said, I can't ping through from EPG to EPG once the service insertion was applied to the contract. (Ping was going thru before.)

Any insight would be greatly appreciated. Thanks! MM

Sergiu.Daniluk · ‎07-11-2020

Hi @Micheline Murphy

Have you performed some captures on ASA? Do you receive the traffic on it?This is just to confirm that redirection is happening.

Also, how does the redirect policy looks like if you check it on CLI of the leaf?

show zoning-rule | grep redir
show service redir info group [redir destgrp value from prev cmd]

Cheers,

Sergiu

Micheline Murphy · ‎07-13-2020

Hello Sergui--thanks for getting back to me. I thought I had the rules configured right, but it turns out that, I've got this going on:

cnat-apic-02# fab 101 show zoning-rule scope 3080192
----------------------------------------------------------------
Node 101 (cnat-nexus9348gc-fxp-1)
----------------------------------------------------------------
+---------+--------+--------+----------+----------------+---------+---------+------+------------------+----------------------+
| Rule ID | SrcEPG | DstEPG | FilterID | Dir | operSt | Scope | Name | Action | Priority |
+---------+--------+--------+----------+----------------+---------+---------+------+------------------+----------------------+
| 4121 | 0 | 0 | implicit | uni-dir | enabled | 3080192 | | deny,log | any_any_any(21) |
| 4120 | 0 | 0 | implarp | uni-dir | enabled | 3080192 | | permit | any_any_filter(17) |
| 4137 | 0 | 15 | implicit | uni-dir | enabled | 3080192 | | deny,log | any_vrf_any_deny(22) |
| 4118 | 0 | 49153 | implicit | uni-dir | enabled | 3080192 | | permit | any_dest_any(16) |
| 4126 | 0 | 16387 | implicit | uni-dir | enabled | 3080192 | | permit | any_dest_any(16) |
| 4123 | 16388 | 49154 | 23 | bi-dir | enabled | 3080192 | | redir(destgrp-7) | fully_qual(7) |
| 4138 | 0 | 16389 | implicit | uni-dir | enabled | 3080192 | | permit | any_dest_any(16) |
| 4112 | 16394 | 16388 | 23 | uni-dir | enabled | 3080192 | | permit | fully_qual(7) |
| 4119 | 49154 | 16388 | 23 | uni-dir-ignore | enabled | 3080192 | | redir(destgrp-8) | fully_qual(7) |
| 4108 | 16393 | 49154 | 23 | uni-dir | enabled | 3080192 | | permit | fully_qual(7) |
+---------+--------+--------+----------+----------------+---------+---------+------+------------------+----------------------+

cnat-apic-02# fab 102 show zoning-rule scope 3080192
----------------------------------------------------------------
Node 102 (cnat-nexus9348gc-fxp-2)
----------------------------------------------------------------
+---------+--------+--------+----------+---------+---------+---------+------+----------+----------------------+
| Rule ID | SrcEPG | DstEPG | FilterID | Dir | operSt | Scope | Name | Action | Priority |
+---------+--------+--------+----------+---------+---------+---------+------+----------+----------------------+
| 4101 | 0 | 0 | implicit | uni-dir | enabled | 3080192 | | deny,log | any_any_any(21) |
| 4105 | 0 | 0 | implarp | uni-dir | enabled | 3080192 | | permit | any_any_filter(17) |
| 4106 | 0 | 15 | implicit | uni-dir | enabled | 3080192 | | deny,log | any_vrf_any_deny(22) |
| 4102 | 0 | 49155 | implicit | uni-dir | enabled | 3080192 | | permit | any_dest_any(16) |
| 4104 | 16394 | 16388 | 23 | uni-dir | enabled | 3080192 | | permit | fully_qual(7) |
+---------+--------+--------+----------+---------+---------+---------+------+----------+----------------------+

Both my consumer and my provider EPGs are on Leaf 101, so physically, what I'd expect the traffic to do is to go Leaf 101--> ASA --> Leaf 102 --> Spine x --> Leaf 101. A bit sub-optimal, I know, but so is not working. I pre-tested the contract config before I started configuring the service insertion, so I know that without the service insertion, VMs in the consumer and provider EPG can reach each other.

I've attached screenshots of the deployed service graph. Also screenshots of the PBR policy.

Very interested to know what you think. Thanks again for your help. MM