ACI Default Static Route to FW - Help

[SJB0095]

Hi,

ACI: we are about to implement ACI and have stumbled across a problem - hoping someone here may be able to help.

We have a multipod setup over two sites connected with dark fibre, a checkpoint HA pair one at each site along with an internet breakout.

I was planning to connect these firewalls to a BD in ACI but need to be able to point the networks default route the the firewalls VIP address (for fail-over) and distribute this to over a separate WAN connection EIGRP.

is this currently possible? how????? :)

[Manuel Velasco · Accepted solution]

Hi

One way to accomplish this is by connecting your FWs to an L3out instead of using EPG static ports and use static routing on the L3out.

[SJB0095]

@Manuel Velasco Thanks for the reply,  

I need a default route pointing to the firewalls VRRP address as they are a High availability pair.

0.0.0.0 -> 10.1.1.200    Firewall VIP (vrrp) address

Firewall VRRP: 10.1.1.200

Firewall 1: 10.1.1.210

Firewall 1: 10.1.1.220

[Manuel Velasco]

Yes, you could also add a static default route to your FW. The screenshot on my last message is to show you where you would add a static route under the L3out.  Note that this need to be added on each leaf node part of this L3out.

[SJB0095]

@Manuel Velasco

Thanks again for the reply, so the static route can still be set even though it will be on both interfaces?

VRRP address shared between the two PODS? 

Firewall VRRP: 10.1.1.200 - Both PODS for fail-over (VRRP address Virtual IP)

Firewall 1: 10.1.1.210 - POD1

Firewall 1: 10.1.1.220 - POD2

So each firewall interface need to talk to each other for VRRP

[Manuel Velasco]

Yes, that is correct which means that we will have two routes for the default route, one from the local pod and one via mp-bgp, but should only have an ARP entry to the active firewall.

[paul.s.lewis]

Will the VRRP work between the firewalls if the only connectivity between them is on the Layer3 outs on the leafs in the two PODs, I though communications for the external EPG only worked if they were both connected on the same leaf 

[SJB0095]

@Manuel Velasco  Hi Manuel, do you know if this is correct?

[Manuel Velasco]

As long as both of your firewalls are part of the SAME L3out the external L3out EPG would work (see below typologies with first lab tested code version)

ACI code 2.2(2)

ACI code 2.3(1)

[SJB0095]

@Manuel Velasco - perfect Thanks for your advice we will try this. 

[SJB0095]

@Manuel Velasco

Hi,

We have been testing this setup L3out for Firewalls with two cisco routers running VRRP and they are unable to communicate with each other over ACI L3out.

Example:

Router1 10.10.10.1 VRRP: 10.10.10.254

Router2 10.10.10.2 VRRP: 10.10.10.254

Pings between routers does not work but both can ping the local leaf and secondary address no the L3out. VRRP remains master on both routers (example Router1 10.10.10.1 unable to ping 10.10.10.2)

We have tested VRRP works out side of ACI.

Router output:

Router-1#sh vrrp brief

Interface          Grp Pri Time  Own Pre State   Master addr     Group addr

Vl10               1   50  3804       Y  Master  10.10.10.1      10.10.10.254

Router-2#sh vrrp brief

Interface          Grp Pri Time  Own Pre State   Master addr     Group addr

Vl10               1   99  3613       Y  Master  10.10.10.2      10.10.10.254

We would really appreciate your input on this

[SJB0095]

@Manuel Velasco quick topology picture if it helps

[Manuel Velasco]

Can one router ping both leaf primary IPs?

Do the routers have arp entries for all of the IPs in the subnet?

Can the leaf nodes ping each other?  Use iping command

“iping -V <IP_Leaf2> <VRF_Name> -S <IP_Leaf1>

Is this multipod set up working for other endpoint between pods?

[SJB0095]

@Manuel Velasco

Leafs:

the leafs can iping each others loopback address on the L3out but not the SVI's to the router.

Routers:

looks like the router is learning only the ARP from the local leaf not the remote leaf, you can only ping the local leaf ip's.

this is a new multipod setup we've tested remote EGP's between pods but this is the first L3out

[Manuel Velasco]

You created a single L3out for both of this routers, correct?

What do you mean by “the leaf can iping each other’s loopback”, what about the SVIs defined the in the L3out logical interfaces?

If the arp table doesn’t have all the entries that explains why communication between the routers is failing.  It might also means that ARP packets are not making it accross the multipod/IPN setup, so I would double check the multicast configuration on the IPN.

Have you verified if the arp packets are making it to the other router?