Dual cloud dmvpn as a backup for mpls

[umutyasar]

Hi,

I'm testing a dual cloud dmvpn as a backup for mpls. Dmvpn using ibgp and mpls using ebgp between hub and spokes. Each Cloud has single hub and these two hubs sharing same DC LAN Network subnet  and using eigrp in DC.

Each huB has one mpls link and one dmvpn tunnel. Each spoke has one mpls link and one tunnel to dmvpn1 and one tunnel to dmvpn2. I want hub1 mpls primary, hub2mpls secondary, hub1Dmvpn third and hub2dmvpn Last to  take over the traffic as redundancy. But once  both mpls links goes down on hubs or, mpls uplink goes down on spoke, spoke egress trafic uses dmvpn1 and ingress traffic uses dmvpn2. Is there any way to prevent this and force ingress traffic to use dmvpn1 as well?

[Giuseppe Larosa · Accepted solution]

Hello umutyar,

your network scenario is quite complex and a network diagram would help. Can you post a network diagram as an attachment file?

Can you explain how the two DMVPN clouds are implemented ?

Each device is using an MPLS link and two internet facing links from different providers one for DMVPN1 and one for DMVPN2?

Or both the DMVPN clouds use the same internet facing link?.

>> Dmvpn using ibgp and mpls using ebgp between hub and spokes. Each Cloud has single hub and these two hubs sharing same DC LAN Network subnet and using eigrp in DC.

Can you provide the configuration of:

DMVPN1 Hub1

DMVPN2 Hub2

and one or two Spokes

again as attachment files in text format.

It is not clear to me how you can implement eBGP sessions and iBGP sessions between the same set of devices

I can guess you are using neighbor local-as command and its variations.

Note: unless the MPLS service is a layer3 VPN and you are doing eBGP with the PE node of the MPLS provider on each device. This is also another point that needs to be clarified.

>> But once both mpls links goes down on hubs or, mpls uplink goes down on spoke, spoke egress trafic uses dmvpn1 and ingress traffic uses dmvpn2. Is there any way to prevent this and force ingress traffic to use dmvpn1 as well?

You are using iBGP, eBGP and EIGRP on the hub routers.

The upstream traffic from spoke to DMVPN1 is chosen on spoke using iBGP (likely an higher weight or local preference is given to iBGP route coming from HUB1 on DMVPN1 when compared to route received from HUB2 on DMVPN2).

The return traffic is chosen by EIGRP protocol in the data center.

How do you redistribute iBGP routes into EIGRP in HUB1 and HUB2?

You need to use route-maps, but if you want to be sure that HUB1 is also used in return path you need to tune EIGRP metric used in redistribution

Example:

HUB1

router eigrp 100

default-metric 20000 100 255 1 1500

redistribute bgp 65000 route-map BGP-into-EIGRP

! on Hub2 you need to use a different seed metric

Hub2:

router eigrp 100

default-metric 10000 1000 255 1 1500

redistribute bgp 65000 route-map BGP-into-EIGRP

see the EIGRP command reference. The first value is bandwidth and the second is delay. The other parameters are not important with default K values in EIGRP

https://www.cisco.com/c/m/en_us/techdoc/dc/reference/cli/nxos/commands/eigrp/default-metric-eigrp.html

To be noted the route-maps BGP-into-EIGRP must take in account if the routes are learned via eBGP or via iBGP and set a seed metric accordingly. This can be done by checking the BGP next-hop.

This is just a suggestion on what should be done.

If you provide the configuration details of the Hub1, Hub2 devices and one Spoke it is possible to provide more focused suggestions.

Hope to help

Giuseppe

[umutyasar]

Hi Giuseppe,

Thank you for your reply.

I think you got what I need in your comment below:

>>"To be noted the route-maps BGP-into-EIGRP must take in account if the routes are learned via eBGP or via iBGP and set a seed metric accordingly. This can be done by checking the BGP next-hop."

I was trying to do it for some time but I couldn't find a way how to check BGP next-hop and apply different metrics to EBGP learned routes and IBGP learned routes in Hub1 and Hub2.

I want to apply for instance like below, I know I need to check next-hop (ebgp/ibgp learned) and apply seed metrics accordingly in a route map since two commands can not be active at the same time in a router, I wrote  it as a reference for seed metric sequence for redundancy I want to use. But I couldn't figure out how to do it in cisco router. I can use default metric as well as you mentioned in your email, but again how to change seed metrics according to next-hop in a route map?

Could you give an example config?

HUB1

EIGRP 100

redistribute bgp 65000 metric 10000 1 255 1 1500 route-map BGP_TO_EIGRP (for EBGP primary MPLS link)

!

redistribute bgp 6500 metric 10000 200 255 1 1500 route-map BGP_TO_EIGRP (for IBGP DMVPN01 tunnel)

HUB2

EIGRP 100

redistribute bgp 65000 metric 10000 100 255 1 1500 route-map BGP_TO_EIGRP (for EBGP secondary MPLS link)

!

redistribute bgp 6500 metric 10000 1000 255 1 1500 route-map BGP_TO_EIGRP (for IBGP DMVPN02 tunnel)

Regards,

Umut

[umutyasar]

Hi Giuseppe,

I think I figured it out.

I tried the route-map and it's working. I didn't use default-metric instead I used what I mentioned previously to set metric and just edited it for IBGP learned routes with higher seed metric as I mentioned before. I think this is just another way of setting a metric. But I'll change it to default-metric as I read it's best-practice when redistributing the routes.

I used 10000 for min bandwidth for all route map and just changed delay, different from your example. I think it's ok as well, just used delay for changing the distance calculation.  I reckon this metric is nothing to do with actual min bandwidth of the route as it's for redistribution.

Regards,

Umut

[Giuseppe Larosa]

Hello Umut,

to check the BGP next-hop in a route-map you need to use the match ip next-hop <standard-ACL>

example:

your eBGP next-hop on MPLS is IP address 172.16.55.22 on Hub1

access-list 11 remark check eBGP next-hop

access-list 11 permit host 172.16.55.22

All iBGP routes learned over the DMVPN1 will have different next-hops

We can use a prefix-list to match all possible spoke routes

ip prefix-list Spoke-Routes permit 172.25.0.0/16 le 32

The route-map BGP-into-EIGRP will have two blocks

route-map BGP-into-EIGRP permit 10

match ip next-hop 11

match ip address prefix Spoke-Routes

set metric 10000 100 255 1 1500

route-map BGP-into-EIGRP permit 20

match ip address prefix Spoke-Routes

set metric 10000 1000 255 1 1500

You can use a single route-map to redistribute from BGP into EIGRP, but the route-map can  be made of multiple block or clauses.

On Hub2 the eBGP next-hop will be different like 172.16.55.155/32

You will use a different ACL to be invoked in match ip next-hop statement

access-list 12 remark match eBGP next-hop HUB2

access-list 12 permit host 172.16.55.155

The same prefix-list can be used

We can use a prefix-list to match all possible spoke routes

ip prefix-list Spoke-Routes permit 172.25.0.0/16 le 32

The route-map will become:

The route-map BGP-into-EIGRP will have two blocks

route-map BGP-into-EIGRP permit 10

match ip next-hop 12

match ip address prefix Spoke-Routes

set metric 10000 200 255 1 1500

route-map BGP-into-EIGRP permit 20

match ip address prefix Spoke-Routes

set metric 10000 2000 255 1 1500

The key point is that in eBGP the BGP next-hop is rewritten and it is equal to the eBGP peer address.

In iBGP the next-hops will be unchanged and will belong to each flat network in DMVPN1 and in DMVPN2 respectively.

It is enough to check if the route has a BGP next-hop equal to the expected eBGP next-hop to assign it the lower metric in clause permit 10.

The use of a prefix-list allows to control what you are redistributing into EIGRP.

If a single route coming from a specific prefix is not advertised anymore in eBGP HUB1 will change its seed metric and so Hub2 will do and Hub1 will inject a better route and return traffic will be via DMVPN1 as desired.

Hope to help

Giuseppe

[umutyasar]

Hi Giuseppe,

Yes, As I mentioned in the previous reply  I created a very similar access-list and route map as yours after I asked you :)

Different from yours I used default-metric for the EIGRP and for the IBGP routes I used access-list as well for the next-hop.

Sometimes when a person asks the question he finds the answer afterward by himself.

I tested it and it's working. But thank you for your reply anyway.

The prefix-list for spoke-routes is a good idea to control redist to eigrp, I plan to use this.

Regards,

Umut Yasar

[umutyasar]

Hi Giuseppe,

After testing several times I found out that when Hub1 MPLS and Hub2 MPLS links both down, the return path from DC choosing Hub2, even if it has a higher metric.

Once I clear both Hub1 and Hub2 Eigrp neighbors, Hub1 becoming return path.

Do you have any idea about the cause of this?

Below is from DC router once the Hub2 become return path

P 192.168.x.0/24, 1 successors, FD is 768256, tag is A
via a.a.a.1 (768256/768000), VlanZ

Below is from DC router once the Hub1 become return path after resetting eigrp neighbor

P 192.168.x.0/24, 1 successors, FD is 512256, tag is A
via a.a.a.2 (512256/512000), VlanZ, serno 2361

[Giuseppe Larosa]

Hello Umut,

I agree it is strange.

Can you provide the following output taken from DC router

show ip eigrp topology 192.168.x.0 all-links

a) in normal conditions with both MPLS links up on Hub1 and Hub2

b) just after both MPLS links fail on Hub1 and Hub2

c) after you reset the EIGRP neighborships on DC router

Also how are made your route-maps have you two blocks /clauses one for eBGP next-hop alive and one for iBGP like I have proposed ?

From the output you have provided we see that Hub1 provides the lowest metric. But you needed to reset neighbors on DC router to have it realize this.

I wonder if there is any chance the route becomes active after both MPLS links fail on Hub1 and Hub2.

Again your network scenario is quite complex.

Hope to help

Giuseppe

[umutyasar]

Hi Giuseppe,

I solved the problem.

The problem is about the ADs of protocols.

EBGP:20

EXT EIGRP:170

IBGP:200

Once the Hub1 MPLS Down, Hub2 MPLS becomes primary. It gets the route from EBGP and redist it into EIGRP.

Thus HUB1 learns it via EIGRP since its AD is lower than IBGP.

Once HUB2 MPLS becomes down as well, the route is still on its Routing Table, It continues to redist it to EIGRP and HUB2 continues to learn it via EIGRP. Thus one redist is possible, DC router doesn't have a chance to learn lower seed metric route from HUB1.

Solution: I lowered IBGP AD below Ext EIGRP to force HUB1 to learn it from IBGP. Thus the process becomes opposite and HUB1 can redist it to EIGRP. Then DC router learns it from HUB1.

I used this technic before to solve the same problem but without the route-map for seed metric, it didn't work properly. Now a combination of two technics resolved my problem.

Thank you for the EIGRP command, it made me sure that seed metrics configured and redistributed properly for EBGP and IBGP learned routes.

Thank you for your replies.

Regards,

Umut Yasar

[Giuseppe Larosa]

Hello Umut,

thanks for your feedback.

Yes the different routes have different default AD values.

If I have correctly understood you have lowered iBGP AD in router bgp on Hub1 and Hub2 so that an iBGP route is preferred over a D EX EIGRP external route and this solves the issues you have seen so now DC router can see the lower seed metric from Hub1 without the need of clearing EIGRP neighbors.

Combining route-maps with different seed metric for EIGRP and tweaking the AD values made possible to achieve the desired behaviour.

Best Regards

Giuseppe