Re: VxLAN EVPN/BGP campus architecture

ulasinski · ‎11-22-2018

I am sorry for my language at the beginning...

From IOS-XE 10.9.2 VxLAN EVPN/BGP VTEP is implemented in Cat9300/9400/9500.

It's probably the result of competition moves. Arista and Mellanox are pushing the new model of campus based on VxLAN EVPN / BGP.

I would like to ask: in your opinion, what the architecture of the Cisco campus VxLAN EVPN/BGP network should look like?
Is

Should the campus access leafs should have the same spines as dc leafs?

Should the connection be carried out through dedicated borders leafs on both networks?

Or maybe both networks spines (separated) should be connected to super spines?

My adventure
Half a year ago I have deployed in DC N9300s with 100G uplinks based on VxLAN EVPN / BGP - Programmable Fabric (all N9300s support ACI which I could buy).

The time has come to upgrade the campus. I bought Cat9500s for aggregation and Cat9300s for access, but also without APIC. I wanted to run LISP with Geo-VXLAN by CLI. There were huge problems.

The most important for me:

I have networks which I am not allowed to touch inside and I have to have a connection L2(vlan) from DC to the campus. In the new campus model (Campus Fabric) as far as I know today it is't possible. Between you must have L3 connection.

Someone from Cisco told me that you could build an EPG relationship to SGT with APICs in both networks (but he was't sure).

For a month and a half I tried to connect these networks with L2 by port-to-port L2 trunk as physically separate networks without lack.

Also dynamic routing on the port in the anycast-gateway mode between DC and Campus with the L2 connector in the redundant systems is not understandable to me (probably not recommended and nowhere described).

Running the Fabric Campus itself is a feat for me of course :) - for today I can't run DHCP.

Instructions and translations to run Fabric Campus are weak. I have the impression that it would force you to buy APIC.

Now it's possible to attach the Cat9300 directly to the N9K spins witch VxLAN EVPN/BGP. I launched it within two days of work. Work good.

Only what worries me is that I can put away two pieces of cat9500-32c on the shelf. Unless Cisco would let the catalyst be SPINE.

ulasinski · ‎11-29-2018

What do you think about that kind of connection between dc and campus?

The plus for me is that the whole structure is homogeneous.

Border's switch do only ip forwarding and terminal macsec link from campus access switches.

rajesh197475 · ‎02-22-2019

Hi

Can you send some diagram with cisco products used to build campus or enterprise SDN ( What products have to used ).I have idea about SDN datacenter.

I want to design campus SDN switching and also complete SDN network in campus or enterprise.

rajesh197475 · ‎02-22-2019

Hi

Can you send some diagram with cisco products used to build campus or enterprise SDN ( What products have to used ).I have idea about SDN datacenter.

I want to design campus SDN switching and also complete SDN network in campus or enterprise.

ulasinski · ‎02-23-2019

to build a network in middle size company as a whole based on VXLAN (Cisco VXLAN as a homogeneous company network (DC + Campus)), without the use of graphical Cisco managers:

- in DC without ACI (APIC-EM) or DCNM
- in Campus without DNA (APIC-EM)

Remember that with VXLAN in DC you have the choice:
- ACI - based on VXLAN BGP EVPN
- Cisco Programmable Fabric - based on VXLAN BGP EVPN

With VXLAN in CAMPUS you have the choice:
- DNA - based Geo/VXLAN and LISP
- Campus Fabric - based Geo/VXLAN and LISP
- and NEW - VXLAN BGP EVPN

ACI is not the same as Cisco Programmable Fabric.

DNA is not the same as Campus Fabric.

Check on Cisco websites for more information.

Default and recommended solution by Cisco in native configuration with CLI (DCNM) is Cisco Programmable Fabric in DC and Campus Fabric in CAMPUS.

But this solution has huge drawbacks for me. Namely:

Geo / VXLAN and VXLAN are not the same and there is no homogeneous combination of these structures.
So far, there is no dedicated LISP GeoVXLAN - MBGP VXLAN gateway. Cisco recommends native L3 connection between these fabrics.
lack of L2 connections between campus and DC (I failed to run it).
Hard to run DHCP relay from CAMPUS to DC.

It is highly likely that competitive solutions have forced Cisco to run competitions for the widely promoted LISP in CAMPUS.

From version IOS XE 16.09.01 in Catalyst 9300/9400/9500 VXLAN BGP EVPN is supported. This allows inserting these switches in the SPINE - LEAF architecture as LEAF and NEXUS 9000 as SPINE.

It is quite a nice idea to close both networks in one solution. At the same time, you gain all the advantages of the SPINE-LEAF architecture in the CAMPUS network.

Tested hardware:

SPINEs: Nexus 9363C + LAN Enterprise License
DC's LEAFs: NEXUS 93180YC-EX + LAN Enterprise License
CAMPUS LEAFs (EDGEs): Catalyst C9300-48UXM + ADV License

ulasinski · ‎02-23-2019

after 2 months I wouldn't recommend this solution in the production environment because:

- there is a problem with the functioning of EVPN on the Cat9300 platform in the configuration stack (in same part of switches work, and on same part not. Configuration is the same.)
- software16.09.01 seems to be unstable. I have a problem with hanging ports, ports with only PoE and even switch reboots.
- Cisco support is none (Over a month ago I req Tac with EVPN and zero reaction from Cisco TAC - priority 2). I intend to escalate him very much after the holidays in my country because it is a mockery. I take it as a lack of Cisco support skills