topic Re: L3 port-channel between Borders & Fabric nodes in Cisco Catalyst Center

L3 port-channel between Borders & Fabric nodes

waleedmatter — Thu, 26 Aug 2021 18:54:21 GMT

The customer asked the below for example to increase the BW

fabric nodex---------L3 port-channel(2x links)---------Border 1-----------EBGP------Fusion

---------L3 port-channel(2 x links)---------Border-2-----------EBGP------Fusion

And the DNAC is out of the fabric so to discover the fabric , we will use the routed mode and ISIS configuration between the fabric nodes and the Border-1 & 2 and then EBGP between the two borders and the Fusion so my question can i configure L3 port-channel between the fabirc-nodes and the Borders and then apply the ISIS configuration between them manually or the L3 port-channel between them is not supported

Re: L3 port-channel between Borders & Fabric nodes

rasmus.elmholt — Sat, 28 Aug 2021 02:42:10 GMT

Hi,

You can make the underlay as you want, the only important thing for the fabric is L3 routing between the Loopback interfaces on all the nodes(FE->BN). So yes, you could make a L3 LAG between the FE and the BN and run ISIS on that.

However I would personally just configure all 4 links as routed links and then let my ECMP handle the load balancing, since it IMHO does it better than LAG.

But if you do choose to use a LAG remember to tune the load-balancing as the Cat9k is using L2 src-mac(as far as i remember) as the default for load balancing: https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9300/software/release/16-6/configuration_guide/b_166_lyr2_lyr3_9300_cg/b_166_lyr2_lyr3_9300_cg_chapter_011.html#con_1275731

And source MAC could not make sense on a L3 LAG, as all packets have the same source MAC address.

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Sat, 28 Aug 2021 08:21:53 GMT

Thanks for your answer so if i will use normal routed interface dual homed to the border without aggregation ,,the end points which connected to the edge nodes can utilize the two links through the isis when will go to the border , my target to utilize the two links correct ?

fabric nodex---------1 x link ISIS---------Border 1-----------EBGP------Fusion

---------1 x LINK ISIS---------Border-2-----------EBGP------Fusion

Re: L3 port-channel between Borders & Fabric nodes

rasmus.elmholt — Sat, 28 Aug 2021 11:54:15 GMT

Hi,

I thought you would have 4 links in all. 2 from the EN to each BN?

Yes, they will use both borders and links, remember that alle user traffic is encapsulated in VXLAN, so the EN will use both equal cost path from its own Loopback interface to the BNs Loopback interface.

Yes, both BNs will be used to loadbalance traffic, please be aware of asynchronous routing and the support for such on the Fusion Firewall

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Sat, 28 Aug 2021 13:52:25 GMT

I added more links to increase the bw because i think the edge nodes with two to borders will use one link but the isis will make load balance

for the fusion will be cat 9500 and there will be bgp for the global to discover the relay and bgp per vrf for the traffic for each vn and the border will be external to be a default router for the edge nodes

i have another question

two borders——-shared service——fusion

——-employee

——-contract

——non-it

fusion————dc fw(shared services)

————dmz fw (internet , voice gw)

my question

between the two borders and fusion woll be bgp per vrf and leaking between the shared services and the other vpn’s

and from fusion to dc fw. ,, vrf shared service from fusion and global from dc fw and i will make ibgp to advertise the shared service subnets correct ?

for the fusion and dmz fw ,,, vrf shared service from fusion and global from dmz fw and i will make default route point to dmz fw and router back in the dmz fw

so i will make the fusion router as a default route for the border by add command neigbour default-orginate in fusion for each neighbor per vrf with border correct ?

Re: L3 port-channel between Borders & Fabric nodes

rasmus.elmholt — Sat, 28 Aug 2021 20:12:15 GMT

Hi,

I am not sure I follow your questions 100% but it seems like you got the essence of it.

The BN to Fusion connection is like any other VRF-Lite setup. Each VN on each VLAN.

The scenarios I have done the border nodes, handed the traffic of in each VRF/VN/VLAN towards the fusion firewall, and on the firewall it was all in the Global routing table. Or if you want, you can keep one of the VRF(Guest normally) in its own VRF/context.

You can even do route leaking just at you would in a normal VRF-Lite setup.

https://www.cisco.com/c/en/us/support/docs/cloud-systems-management/dna-center/213525-sda-steps-to-configure-fusion-router.html

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Sat, 28 Aug 2021 21:33:29 GMT

Thanks for your answer but i means i have the below

Edge nodes-----------2x Borders---------Fusion----------DC FW(shared services DHCP , DNS , DNAC)

----------DMZ FW-------Voice GW + Internet routers

2x Borders---EBGP------Shared servcie VN---- Fusion

---EBGP------Employee------------

----EBGP-----Contract-------------

And there will be a leaking between VN's to make the Shared service subnets & Employee & Contract to reach to each other

My questions here

Q1:

from Fusion----IBGP--Shared-service VN-------Global--FW DC (subents of the DHCP , DNAC , NTP)

My question , The DC FW will make IBGP with the fusion to send the shared service subnets as well as to recieve the subnets of the employees & contract and it is ok correct ?

Q2:

from Fusion---Shared Service VN-----Static-----DMZ FW------Voice GW

------Internet routers

My question between fusion and the DMZ FW , how i can make te routing ?

I assumed to use vrf Shared Service from the fusion router and global from DMZ FW and make the static routes in the two direction (from fusion default route point to the DMZ FW under this vrf & from DMZ FW routes back for the employees & Contract subnets) correct ?

Q3:

But the border per vrf (employees , contract ) need default route point to the Fusion if any end point need (employee , contract) to access the internet so i will add command neigbour default-originate from fusion per vrf point to the border to send default route to the border per vrf correct ?

Re: L3 port-channel between Borders & Fabric nodes

rasmus.elmholt — Sun, 29 Aug 2021 13:43:49 GMT

A1: yes, this seems correct.

A2: This is an ok option to configure it that way as well.

A3: Yes this is the way I would do it as well.

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Sun, 29 Aug 2021 14:47:15 GMT

Thanks for your answers,

Another questions:-

Q1:

I have ISE (access port) will connect the Borders direct so i will configure this port as a Layer 2 handoff but this will generate the port as a trunk but it should be access because the ISE port is access how i can fix this problem ?

Q2:

As you see that the Border will be a GW for the outside so i will select it as external or internal + external when i will configure L3 handoff

Please advise

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Mon, 30 Aug 2021 11:34:46 GMT

Sorry for another questions below

Q6:

Q7:

As you see that the Border will be a GW for the outside so i will select it as external or internal + external when i will configure L3 handoff or external will be enough

Re: L3 port-channel between Borders & Fabric nodes

rasmus.elmholt — Mon, 30 Aug 2021 13:37:55 GMT

A6: We are getting into some questions about the design here, and I think you should try and follow the Design guides. You can connect servers in a lot of ways. Configure them in a DMZ on the Fusion Router/firewall, connect them on a port on the borders, that are not part of the fabric, or put them in the central datacenter.

Remember that the ISE servers needs to be accessible in the underlay. And the L2 handoff feature is only for overlay handoff.

Take a look at figure 1 here: https://www.cisco.com/c/en/us/support/docs/cloud-systems-management/dna-center/213525-sda-steps-to-configure-fusion-router.html

A7: take a look at the design guide to see what handoff mode you need: https://www.cisco.com/c/en/us/td/docs/solutions/CVD/Campus/cisco-sda-design-guide.html

In the examples I have seen we only have 1 set of bordernodes and they do all traffic handoff. Hence they are EXTERNAL & INTERNAL.

Take a look at figure 21 and 22: https://www.cisco.com/c/en/us/td/docs/solutions/CVD/Campus/cisco-sda-design-guide.html

You default route in the fabric will be an external BN, and if you only want internal specific routes you need the internal BN feature.

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Mon, 30 Aug 2021 14:30:48 GMT

Remember that the ISE servers needs to be accessible in the underlay. And the L2 handoff feature is only for overlay handoff.

Waleed(said) No , we need the ISE to connect overlay to get the reachability only to be reachable to the DNAC which connected to behind the DC-FW and i think the L2 handoff will be Ok and i asked cisco and they recomended to connect it to the Border but my question here layer2 handoff will generate trunk configuration but ISE data ports is supported only access (not tagging)

For Q7:

External is enough , so the border will be the GW for any destination that any end point need to reach

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Mon, 30 Aug 2021 14:48:25 GMT

Sorry answer for Q7 , I will use the Border for handoff L3 as a Internal + external option to use the lisp for the known subnets because i will get the known subnets from the DC(e.g. DHCP , DNAC )nd normal default route for the internet (unknow subnets)

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Mon, 30 Aug 2021 20:17:32 GMT

For A6:-

Waleed(said) I dont think so , we need the ISE to connect overlay to get the reachability only to be reachable to the DNAC which connected to behind the DC-FW to integrate with it and i think the L2 handoff will be Ok and i asked cisco and they recomended to connect it to the Border directly but my question here layer2 handoff will generate trunk configuration but ISE data ports is supported only access (not tagging) how i can solve this issue ?

Edge Nodes--------Border---------Fusion-------DC FW----(Shared services DNAC , DHCP , DNS)

ISE

A7: I will use the Border for handoff L3 as a Internal + external option to use the lisp for the known subnets because i will get the known subnets from the DC(e.g. DHCP , DNAC )nd normal default route for the internet (unknow subnets) correct ?

Re: L3 port-channel between Borders & Fabric nodes

waleedmatter — Wed, 01 Sep 2021 15:46:22 GMT

Edge Nodes--------Border---------Fusion-------DC FW----( DNAC , DHCP , DNS) , Connect ISE here ?

ISE