OSPF design - IP unnumbered or SVI

[Miguel10]

Hi I have recently started working on a site, where there are 2 Hub sites connecting to many CPE's around the country over Layer 2 WAN links (WAN links accept only 1 Dot1Q tagged VLAN and both sides of the connecting devices have to be trunk, ISP is maintaining the Layer 2 tunnel). We were requested to add the secondary WAN link for redundancy purposes and asked to use OSPF as the routing protocol. At the moment they are using static routes. All are L3 capable switches, and they want the routing to be done over the SVI's. All the hub sites have stacked 9500, and the CPE's have stacked 9300. I was planning to create a VLAN per WAN link as otherwise STP will come in to play and failover will be time taking. But in this design I will end up with lot of IP addresses and managing the mess will be difficult if the number of CPE's increase. I was thinking if using IP unnumbered for all VLAN's by creating a loopback interface in each HUB and CPE, all the WAN links will be P2P over SVI's. Will this design feasible or recommended from design prospective? What challenges I might have and also how the ECMP will work incase of Ip unumbered?

Appreciate our valuable advise in advance.

[MHM Cisco World]

follow

[Giuseppe Larosa]

Hello @Miguel10 ,

depending on the number of branch sites and the type of L2 service you get p2p versus VPLS / EVPN you may need just a single VLAN tag.

Let us suppose you get from Carrier Ethernet / Metro provider  ethernet E-LINE also known as EoMPLS pseudowire that is point to point.

You have a /30 for the current primary link, one /30 for the secondary primary link and then you have shown additional 802.1Q tags subnets VL 900 and VL 901 for the "new" links.

First of all, all links can use

ip ospf network point-to-point to avoid unnecessary DR/BDR election .

Both Cat 9500 SVL  ( VSS)  and Cat 9300 SVL (VSS) act as a single logical device.

Both support up to 4094 VLANs in routed mode.

Using interface command

ip ospf cost    50, 100,  200, 300   on both sides of each link you can build a hierarchy if you want.

Without this command you will get per flow CEF based load balancing over 4 links as OSPF supports by default ECMP Equal Cost Multi Path.

You need a correct address plane for example

First of all you need loopback addresses that must be unique

For example taken from

10.254.0.0/24 each loop wll be /32 and it will be used as OSPF RID

You can use something like

10.250.0.0/16 and you can code the VLAN ID in two bytes for human reading

VL 800 ---> 10.250.80.0.0/30

VL 801 ---> 10.250.80.1.0/30

or you can build a table for the mapping.

10.250.0.0/16 subnetted in /30 subnets you take the 800th line for accomodating Vlan 800, the 801th line for VL 801 and so on.

I have never used ip unnumbered loop0 on links . So your question is interesting.

https://www.ciscolive.com/c/dam/r/ciscolive/us/docs/2020/pdf/DGTL-BRKRST-2337.pdf

Very nice design session but unfortunately does not answer to your question directly but it is very valuable from Nick J Russo.

Using 4 Vlans per site means you can accomodate up to 1000 remote sites, but in reality OSPF limits come out before. In other words your Cat 9500 SVL can support likely up to 100 / 400 neighbors so using the ip unnumbered could give you a factor of 4 in gain in the number of the supported branch sites, but I have never tested it.

If moving to OSPF I would use ECMP not creating a hierarchy with ip ospf cost.

It should be tested in a lab. With numbered links I'm sure that ECMP works fine and provides you ECMP without symmetrical routing .  i.e the return path can be different but no firewalls are involved so this is not an issue at all.

If Scalability is an issue iBGP with multipath or eBGP leaf and spine or EIGRP could be better choices as highlighted in the above presentation.

Hope to help

Giuseppe

[Arshad Safrulla]

hi Guiseppe,

Thanks for the insights. These are very valuable information. 
I think IP unnumbered is used to reduce the IP address complexity, I have seen similar setup being used in the Spine and Leaf underlays. But they are done in the L3 P2P links. Here I think the requirement is to use P2P over the SVI's where SVI's will be used for IP unnumbered. Are you aware of any limitations which might come with SVI's? 

[Miguel10]

@Arshad Safrulla Do you have any Cisco document for using IP unnumbered in Spine and Leaf fabrics? May be I can use that as a baseline.

Also regarding the IP addresses I am not much bothered as these will be only transit P2P links, the reason behind using ip unnumbered is somewhat similar to what you mentioned. 

@Giuseppe Larosa Great info. At a given time I would like each CPE to be connected to one single HUB site only to avoid Firewall complexities. So for example CPE1 - I might limit the ECMP maximum path's to 2 to achieve this. 

[Giuseppe Larosa]

Hello @Miguel10 ,

>>

@Giuseppe Larosa Great info. At a given time I would like each CPE to be connected to one single HUB site only to avoid Firewall complexities. So for example CPE1 - I might limit the ECMP maximum path's to 2 to achieve this. 

In this case you will need ip ospf cost on interfaces using the same cost on both directions on the links with the "secondary" Hub.

see also my previous answer to @Arshad Safrulla actually routed ports consume internally allocated VLANs taken from the IEEE 802.1Q VLAN db  1-4093,  with vlan 4094 reserved for SVL.

Hope to help

Giuseppe

[Giuseppe Larosa]

Hello @Arshad Safrulla ,

as far as I know in Cisco implementation a Cisco routed port is actually emulated using an internal VLAN and then an SVI and an access port with all the filters for L2 protocols. This was true until Cisco 6500/6800.

if this is true also for Cat 9500 series the usage of VLANs from the IEEE db is 4 taken from VLAN db ( 1-4093)  internal VLANs for each remote Branch Cat9300 SVL. One vlan likely 4094 is used by SVL itself.

I have no evidence that in Cat9x00 this has changed.

Cat9500 also is not fully distributed and it has local switching on "columns" so also the choice of cabling is important for Branch to Branch communication.

I have attended to Cisco Cat9500 Arch presentaton at Cisco Live Barcelona in 2020, but I have missed to put this question to the presenter.

Hope to help

Giuseppe