12-09-2021 10:35 PM
We have configured OSPF between FTD firewall and ACI. Also, We are able to form neighborship with FTD. Now the issue is that when We form neighborship we see two best path in Routing table so now it should be load-balancing. Can we do some changes on ACI so that only one best path is selected keeping SVI and VPC topology.
Solved! Go to Solution.
12-11-2021 12:33 PM
Hi @OmkarPol52290 ,
Also, the Firewall is in Act/Std
I wish I'd know that earlier! That changes my advice completely!
To begin, you really don't want to run a dynamic routing protocol on a VPC that is configured with an Active/Standby neigbour. So remove the OSPF configuration.
Instead:
You'll find a better explanation on the Unofficial ACI Guide blog
Now, if you really, really, really want to use OSPF [I don't like this design because one adjacency (on the standby link) probably won't ever form until the primary goes down, and there would be the full adjacency exchange as OSPF re-establishes itself], stick with the point-to-point links, and the two L3Outs. You PROBABLY could use ONE L3Out with two node profiles, or even one node profile with two nodes, but I'd have to research that before I'd give a definite answer - and since I haven't researched it this whole paragraph may be utterly wrong!
12-12-2021 11:56 AM
Hi @OmkarPol52290 ,
We need dynamic routing as it's enabled on all infra and doesn't make sense to keep static just for ACI.
This reminds me of arguments I heard in the 1990s when people insisted that their Solaris workstations had to run RIP "because it's enabled on the network and doesn't make sense to just configure a default gateway"
When ACI was designed, it was designed to more like an "edge appliance" than a "core router". (In fact, v1.0 didn't even support transit routing). So generally having a default static route configured in ACI is not a big deal if there is only one external router. I don't know if your design has other external routers apart from the firewall, so I can't comment.
But, I don't see why we can't get some kind of OSPF design to work for you. The secondary IP address thing is only relevant for static routing, and as for...
Also this secondary IP design always rasises a fault in ACI about packet-dropped. Are you also experiencing it.
I haven't experienced it, but to be honest I've only configured it in a lab situation as a proof of concept for myself. Let's add a call-out to @Sergiu.Daniluk and ask for his opinion - he's had more real-world experience than I have, and I won't be offended if he comes up with a simple and completely different answer to mine
But at the moment, I think your best bet if you want to stick with OSPF is the two point-to-point links.
12-12-2021 10:40 PM
Hi Guys,
Interesting topic. Thanks Chris for tagging me.
First I would like to answer the question of this topic: "Can we do some changes on ACI so that only one best path is selected keeping SVI and VPC topology?" answer is NO. VPC and best path selection is a no-go because of multiple reasons, some of which are already explained by Chris.
Second, I would like to touch this topic: "So the requirement is that by default ECMP will happen which can lead to suboptimal forwarding according to a few of our colleagues due to which we have to make this change".
Could you elaborate what you mean by suboptimal forwarding? By design, ACI Fabric is a CLOS network, meaning from any compute leaf to your FTD, you will have the same number of hops regardless if you have 1 "best" leaf, or 2 active/active leafs (even if they are or not configured in vPC). The advantage of VPC is fault tolerance in case of one link failure between FTD and ACI Leafs.
Only "advantage" in having one "best" leaf is predictability of traffic forwarding/routing. But if I would have to choose between predictability on which interface traffic will be forwarded and fault tolerance, the second one will always win especially in a data center.
If is not clear enough already, my strong suggestion is to avoid path selection and use vPC as-is (if using static routes) or two point-to-point links (if using OSPF simply to avoid DR/BDRs).
If not convinced already to use active/active Leafs, and you still plan on using a "best path" leaf, then I would suggest to use one single L3Out with two logical node profiles, one for each leaf, and in each logical node profile you will have one logical interface profile, with two SVIs (same IP) but different interface - one to Active FW and one to Standby FW.
Last but not least, "Also this secondary IP design always rasises a fault in ACI about packet-dropped. Are you also experiencing it." - I never seen problems or faults with secondary IP designs, unless configured incorrectly. Could be that what you are seeing is not about the secondary IP, but rather about vPC - packet drops (control plane packets like LLDP) between vpc peers which is a cosmetic bug.
Stay safe,
Sergiu
12-09-2021 11:46 PM
Just influence the decision through "ip ospf cost" command to have your best path.
12-10-2021 01:07 AM
But how can we do it for SVI for vPC in ACI? They will have the same interface policy.
12-10-2021 02:15 AM
My first though is "Why do you want to change the way it is designed to work?"
BUT
To answer your question - you'll need to re-design the setup so there is NO VPC - which is not a bad design anyway. But will require an extra IP on the FW.
12-10-2021 02:28 AM
Hey @RedNectar ,
Hope you're doing ok.
So the requirement is that by default ECMP will happen which can lead to suboptimal forwarding according to a few of our colleagues due to which we have to make this change. So, I tried it in Traditional Setup, and I was able to achieve it via
router OSPF 1
maximum-path 1
############################################################################
Router#
*Dec 10 12:27:25.983: %SYS-5-CONFIG_I: Configured from console by consoleconf t
Router#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override
Gateway of last resort is not set
192.168.10.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.10.0/24 is directly connected, FastEthernet0/0
L 192.168.10.1/32 is directly connected, FastEthernet0/0
O 192.168.20.0/24 [110/2] via 192.168.10.102, 00:00:04, FastEthernet0/0
[110/2] via 192.168.10.101, 00:00:04, FastEthernet0/0
Router#
Router#
Router#
Router#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)#router ospf 1
Router(config-router)#maximum-paths 1
Router(config-router)#end
Router#wr
Building configuration...
[OK]
Router#
Router#
Router#
Router#
*Dec 10 12:27:39.807: %SYS-5-CONFIG_I: Configured from console by consoleconf t
Router#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
+ - replicated route, % - next hop override
Gateway of last resort is not set
192.168.10.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.10.0/24 is directly connected, FastEthernet0/0
L 192.168.10.1/32 is directly connected, FastEthernet0/0
O 192.168.20.0/24 [110/2] via 192.168.10.102, 00:00:05, FastEthernet0/0
Router#
#############################################################################
On northbound I have Firepower so is there a way I can achieve it.
12-10-2021 09:21 PM
Hi @OmkarPol52290 ,
I think I see your problem - you are thinking of ACI as "a router"
But it isn't.
EACH LEAF is a router
So, what you REALLY have in the above scenario is something that looks (as far as OSPF is concerned) more like this:
A multi-access OSPF network with three routers, all on the same subnet. This is made possible because when you create a L3Out in ACI and use a VPC as the attachment, ACI creates an internal hidden Bridge Domain that allows the two leaves to see each other and form an OSPF adjacency - with DR and BDR elections. When the external device sends OSPF Hellos, the Hellos are seen by both leaves and each leaf forms an adjacency.
The problem is, since you've configured this as a VPC, each of the interfaces of the VPC are configured exactly the same - you can'd change the OSPF cost on either interface.
Now, from your explanation
ECMP will happen which can lead to suboptimal forwarding according to a few of our colleagues
I'm guessing that there are severs on one or other of the leaves that you'd like to
As for #1 above - don't worry, That will always happen unless the particular leaf looses it's physical connection.
As for #2, ensuring that the return traffic favours one particular leaf, I honestly think you should not care. ACI was designed to be flexible and dynamic, so trying to do push traffic one way or another is not a good idea generally.
However, the customer is always right, so let's assume you really really really want to implement this BAD idea.
Like this
Now you configure two L3Outs, one for Leaf101 and one for Leaf102. If you want the external router to prefer say Leaf102, then increase the cost of the interface in Leaf101.
I hope this helps.
Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem.
12-11-2021 12:29 AM
Hey @RedNectar,
Thanks as always so I'm confused by the Last statement " Now you configure two L3Outs, " can't we use the same L3 out and manipulate the Priority per Leaf as we'll be able to have separate Leaf profiles and Interface profiles.
Also, the Firewall is in Act/Std so we need to deploy it in SVI as Routed Interface won't work.
Also, we did try /30 between Leaf 101/102 to FTD with /30 each but OSPF neighborship was not coming up. We still are looking into that. I doubt maybe This is due to the same Bridge domain but I don't think that could be the issue here.
I'll be online on Monday I'll check and let you know.
Thanks as always.
12-11-2021 12:33 PM
Hi @OmkarPol52290 ,
Also, the Firewall is in Act/Std
I wish I'd know that earlier! That changes my advice completely!
To begin, you really don't want to run a dynamic routing protocol on a VPC that is configured with an Active/Standby neigbour. So remove the OSPF configuration.
Instead:
You'll find a better explanation on the Unofficial ACI Guide blog
Now, if you really, really, really want to use OSPF [I don't like this design because one adjacency (on the standby link) probably won't ever form until the primary goes down, and there would be the full adjacency exchange as OSPF re-establishes itself], stick with the point-to-point links, and the two L3Outs. You PROBABLY could use ONE L3Out with two node profiles, or even one node profile with two nodes, but I'd have to research that before I'd give a definite answer - and since I haven't researched it this whole paragraph may be utterly wrong!
12-11-2021 09:09 PM
Hey @RedNectar ,
First of all really sorry that I forgot to tell you and also add that part in diagram.
We need dynamic routing as it's enabled on all infra and doesn't make sense to keep static just for ACI.
and this secondary IP with vPC I have done this multiple times. But we cannot do this here.
I'll be joining on monday in next 24 hours I'll let you know my findings.
Also this secondary IP design always rasises a fault in ACI about packet-dropped. Are you also experiencing it.
12-12-2021 11:56 AM
Hi @OmkarPol52290 ,
We need dynamic routing as it's enabled on all infra and doesn't make sense to keep static just for ACI.
This reminds me of arguments I heard in the 1990s when people insisted that their Solaris workstations had to run RIP "because it's enabled on the network and doesn't make sense to just configure a default gateway"
When ACI was designed, it was designed to more like an "edge appliance" than a "core router". (In fact, v1.0 didn't even support transit routing). So generally having a default static route configured in ACI is not a big deal if there is only one external router. I don't know if your design has other external routers apart from the firewall, so I can't comment.
But, I don't see why we can't get some kind of OSPF design to work for you. The secondary IP address thing is only relevant for static routing, and as for...
Also this secondary IP design always rasises a fault in ACI about packet-dropped. Are you also experiencing it.
I haven't experienced it, but to be honest I've only configured it in a lab situation as a proof of concept for myself. Let's add a call-out to @Sergiu.Daniluk and ask for his opinion - he's had more real-world experience than I have, and I won't be offended if he comes up with a simple and completely different answer to mine
But at the moment, I think your best bet if you want to stick with OSPF is the two point-to-point links.
12-12-2021 10:40 PM
Hi Guys,
Interesting topic. Thanks Chris for tagging me.
First I would like to answer the question of this topic: "Can we do some changes on ACI so that only one best path is selected keeping SVI and VPC topology?" answer is NO. VPC and best path selection is a no-go because of multiple reasons, some of which are already explained by Chris.
Second, I would like to touch this topic: "So the requirement is that by default ECMP will happen which can lead to suboptimal forwarding according to a few of our colleagues due to which we have to make this change".
Could you elaborate what you mean by suboptimal forwarding? By design, ACI Fabric is a CLOS network, meaning from any compute leaf to your FTD, you will have the same number of hops regardless if you have 1 "best" leaf, or 2 active/active leafs (even if they are or not configured in vPC). The advantage of VPC is fault tolerance in case of one link failure between FTD and ACI Leafs.
Only "advantage" in having one "best" leaf is predictability of traffic forwarding/routing. But if I would have to choose between predictability on which interface traffic will be forwarded and fault tolerance, the second one will always win especially in a data center.
If is not clear enough already, my strong suggestion is to avoid path selection and use vPC as-is (if using static routes) or two point-to-point links (if using OSPF simply to avoid DR/BDRs).
If not convinced already to use active/active Leafs, and you still plan on using a "best path" leaf, then I would suggest to use one single L3Out with two logical node profiles, one for each leaf, and in each logical node profile you will have one logical interface profile, with two SVIs (same IP) but different interface - one to Active FW and one to Standby FW.
Last but not least, "Also this secondary IP design always rasises a fault in ACI about packet-dropped. Are you also experiencing it." - I never seen problems or faults with secondary IP designs, unless configured incorrectly. Could be that what you are seeing is not about the secondary IP, but rather about vPC - packet drops (control plane packets like LLDP) between vpc peers which is a cosmetic bug.
Stay safe,
Sergiu
12-13-2021 04:19 AM
Hey @Sergiu.Daniluk,
Thanks for your reply.
It's as you suspected - The only "advantage" in having one "best" leaf is the predictability of traffic forwarding/routing. it's not for the underlay part. it's more from the end of the FTD firewall as it'll have the two best paths to go towards ACI fabric. So sometimes traffic will go from Leaf 101 to Leaf 102 a few times. as FW will send a packet to Leaf101 and the Server might be connected on Leaf 102.
As you said we have already configured it but the neighborship is not coming up but that is something we have to check.
I completely agree with you and Chris both but We cannot argue with the application team, for some reason they have a problem with this.
no issues as always thanks, guys.
I'll update you once I get the results.
As for that packet dropped Yes, it's like you said the description says packet-dropped. F1545 and 1547 (Is this it).
Also, I know I'm asking a lot but we see traceroute as sometimes it shows both the IPs for Leaf
as let's say we have Leaf 1 (.11) and Leaf 2 (.12) and VIP is (.10) then we are observing trace after the FW hop as (.11) and (.12).
Note - This setup has Secondary IP configured with vPC with FW pointing at Sec IP.
12-13-2021 05:08 AM
It's expected to see both leaf in traceroute if the traffic arrives on Leaf1 but the destination EP is single homed on Leaf2.
Regarding the faults, have a look on this: https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuz99892?rfs=iqvred
Its cosmetic - just ignore/squelch the fault.
"We cannot argue with the application team, for some reason they have a problem with this." -> in my opinion, if the problem they have is not sustained with technical arguments, then it's invalid. In the end, the network is configured for applications, so it's network engineer to decide what is the best/optimal design.
Anyway, I hope all the details provided by Chris and I will help you and your team to take the best decision for the final implementation.
Stay safe,
Sergiu
12-13-2021 05:24 AM
Hey @RedNectar and @Sergiu.Daniluk
Thanks for your help.
I'll update you guys on progress once I have the complete this.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide