DNA BORDER in two different building connectivity same fabric

[mujeeb2005]

Hi,

I have three building in a single fabric domain. each building has two borders connected to the Fusion,

i would also like to have direct connection between the borders of three building, present they are connected through the Fusion.

Is it possible to have this connection as part of the underlay network.

any best practise, of connecting the Borders in two building 

[jedolphi]

Hello mujeeb, please clarify:

Q1. We have the concept of fabric domain and also fabric site. A fabric domain can contain multiple fabric sites. In your scenario, is each building mapped a unique fabric site? Or are all buildings combined into a single fabric site?

Q2. You have a total of 6 borders, correct? What type of borders are they? Internal, or external, or internal+external?

Best regards, Jerome

[mujeeb2005]

Hi Jerome,

Thanks,

I have three buildings part of the same site.

All the 6 borders will be connected to the fusion(C6807), 

All the border are internal+external,

See, i added all the border as part of the same building, i am able to configure the back to back link between the borders through LAN automation as part of the underlay

but when i place the Borders in different building, DNA is not configuring the B2B link as part of the underlay.

Ok why i want this, i don't want the inter-building traffic to reach the fusion.

Is it recommended, to have iBGP between the different building border

Regards,

[jedolphi]

Hi Mujeeb,

A few notes for your consideration:

*There must be some error here, DNA Center will not allow any fabric site to have more than 4x external borders. (internal+external border counts as an external border). So actually it is impossible to have 6x internal+external borders

*Has this SD-Access network already been implemented or are you designing a new network? I think it must be in design phase since as per my previous point it's not possible to have 6x internal+external borders

*Is anyone from Cisco helping with the design? Selecting the location and type of borders is important in design phase, ideally you don't want to be changing the border types after implementation. Do you have a Cisco SE or AM you can contact? I would recommend that you contact your SE or AM and ask for some design assistance. You can give them my user ID (jedolphi) and I can work with the sales team to do get a design review organised

*If all buildings are part of same fabric site then inter-building traffic in same SD-Access VN will *never* reach the fusion. Always SD-Access fabric will be used to get from an FE1 (Fabric Edge) to FE2 at same fabric site in same VN

Cheers, Jerome

[mujeeb2005]

Dear Jerome,

Thanks,

it's not yet implemented, i am still in the POC phase. to validate my design

I have found a solution to my problem. (some what)

i have one site with only one building (instead of 3)in the hierarchy, added all the border into the same building,

This way, i can have LAN automation configure my P2P link between all my Borders.

Now, as there is limitation on internal+external of max 4, 

can i have 4 as  internal+external, and 2 as internal, as i see they import and export the routes from outside.

Please also share, if any useful documents regards to designing

Regards, 

[jedolphi]

I don't know your customer or the design requirements so what I write here is general advice.

An internal border is usually used to connect SD-Access fabric to a routing domain that is separate from the routing domain connected to the external borders, for example:
*External border connects to WAN and Internet, OR, external border connects to a network that in turn has connectivity to the whole of the rest of your company
*Internal border connects to a different routing domain (maybe a data centre for example) with unique prefixes not reachable via the external border

Usually if all prefixes that are external to SD-Access fabric are reachable by the same egress path, we tend to try to use only external border, not internal+external border. Often an external border is simpler to understand and operate than an internal+external border. All traffic destined for prefixes unknown to the SD-Access control plane is sent to the the external border. You can think of the external border as effectively being a default gateway for the SD-Access fabric.

An internal border will not import a default route from your external routing domain, so unless you will be connecting internal borders to a different routing domain than your external borders, it does not make sense to use 4xinternal+external + 2xinternal.

There is scenarios where having 6 borders on a fabric site is necessary, but generally speaking it's not a highly recommended design. Why do you need borders in each building? If an endpoint in building 1 wants to talk to a server external to SD-Access fabric, and the endpoint is connected to B1FE1 (building 1 fabric edge switch 1) then B1FE1 might choose to use a border in building 2 (B2BDR1) to egress the fabric, in the vast majority of scenarios you wont be able to force B1FE1 to use a border in building 1. So having borders in each building most likely doesn't accomplish anything. In addition, if each building is interconnected by mulitple diverse high speed switch-to-switch links (and they must be if all 3 building are part of the same fabric site) then having borders in each building isn't necessary, since we are talking about tiny fractions of a second in latency to forward a packet from B1FE1 to B2BDR1.

Again, I do strongly recommend you contact your Cisco AM or SE to kick off a design review. The AM or SE can contact me directly or any SD-Access expert inside of Cisco. Because SD-Access is a reasonably new solution suite we prefer to work closely with our customers and partners on their first design/implementation to ensure the designs are optimal and the implementation strategy is optimal. If you don't have an SE or AM then drop me an email directly and I'll find someone for you - jedolphi@cisco.com

Regarding collateral publicly available right now, my answer will change in a few weeks when the Cisco Live Barcelona 2019 materials are released. But as of today:

CVDs, use the ones published reasonably recently, not the older ones:
https://www.cisco.com/c/en/us/solutions/enterprise/design-zone-campus/design-guide-listing.html?dtid=osscdc000283

All of these Cisco Live presentations
https://ciscolive.cisco.com/on-demand-library/?search.event=ciscoliveus2018&search=brkcrs-2811#/session/1509501694123001Pzhd
https://ciscolive.cisco.com/on-demand-library/?search.event=ciscoliveus2018&search=brkcrs-2812#/session/1509501694230001P3bW
https://ciscolive.cisco.com/on-demand-library/?search.event=ciscoliveus2018&search=brkcrs-2810#/session/1509501694005001Pn6t

HTH, Jerome

[Steve Leemk]

So, If I have three buildings all in single fabric, I have server farm in each of the buildings. can I have 4 default borders in HQ(one of the 3 buildings) and two internal borders in each of the remaining buildings?
Total: 4 default borders (HQ) + 4 internal borders (2 remote office buildings)

PS: 4 default borders across 2 DC and 2 internal borders for each remote office are redundancy consideration. No single point of failure in each building

[jedolphi]

Hello Man Kit Lee,

Yes, that is permitted in the DNA Centre workflows. Please note that internal border will never function as a default gateway for fabric, internal border will be for accessing known internal IP ranges only.

Jerome

[Mike.Cifelli]

Jerome's responses are very helpful.  I just want to add a couple of things based on my SDA experience:

-It is recommended to have the iBGP connection between your borders.  

-I dont see the need to have more than 2 borders.  As long as you have two borders in for example your main building that connect you to your fusions you should be good.

-I recommend building a mesh between your buildings if the physical layer permits you to accomplish this via IN connectivity.  In my SDA fabric I have two fusions, two EBNs, and two INs located in the main building.  The INs have several connections which helps with the mesh piece and overall redundancy between campus buildings.  

-I recommend enabling BFD on your L3 links.

-When determining how you plan to build out your VNs, IP pools, SGTs I think it is crucial to determine how/where you want to manage these things.  What I mean by this is, if you determine that you want most of your departments in totally separate instances, but these instances will require access to one another then know that you will be controlling this via route leaking at your fusions.  If separate instances are not required it may be easier to have less VNs, and rely on separate IP pools/SGTs and trustsec to manage your connectivity/restrictions.  IMO it doesnt make sense for two users connected to the same switch, but in different VNs to have to traverse to fusion to get to each other.

Hope this helps.