OSPF - Summary Routes and ECMP

[danielgoing11]

Hey Cisco Forum,

I'm running a lab for some routing simulations and have come across some weird route summarization that I can't work out an answer to.

My lab looks like this (from a routing perspective at least):

BGP1 (Palo Alto FW)---OSPF Pid120---OSPFPid10 (N9k)---BGP2(3725 Router)
BGP1 (Palo Alto FW)---OSPF Pid120---OSPFPid10 (N9k)---BGP2(3725 Router)

Without summarization, I get ECMP across each AS giving me L3 equal cost load balancing to all the right networks. However, when I summarise the routes, I lose the ECMP.

I aggregate the routes in BGP1, and I see the aggregated routes in OSPF process 120, on both Nexus switch A and Nexus switch B. OSPF process 120 redistributes into OSPF 10. OSPF 10 redistributes into BGP2. Thing is, when I aggregate my routes on BGP1, by the time those routes hit BGP 2 (through OSPF120 and OSPF10), BGP2 no longer sees equal cost path. As soon as I disable the aggregate route, my equal cost path returns. Does anyone know why this is? I don't understand, I've been reading up online and I can't seem to figure out why it is.

None of my OSPF config changes, only that the external route it's learned from BGP1 is a /17 instead of 3 /24's. The aggregation happens in BGP as an export on the Palo Alto firewall, so it's just another external network as far as OSPF is concerned.

I don't understand why I lose the ECMP, and it's bugging the heck outta me!

Thanks in advance all, I can upload configs if it's needed but I have a sneaky suspicion that it's something glaringly obvious I'm missing. (At least I hope it is!)

Best, Dan

[Georg Pauwen]

Hello,

curious...

Can you post the output of 'show ip bgp topology *' for both the aggregate and the non-aggregate route scenarios ?

[danielgoing11]

Hey Georg,

Thanks for jumping in.

The iOS I'm running in the lab doesn't support show ip bgp topology. I think show ip bgp should do it though. I've included some show commands for OSPF too, I think the issue is within OSPF. 

As shown in the output below, with no aggregate routes I see equal cost to 10.125.128.0/19 through show ip route, learned through OSPF, also included show ip ospf database external to see the LSA's. When I enable the aggregate route out in BGP1 (HA - palo alto firewall), I only see a single path to 10.125.128.0/17 in the routing table of RTR1 and RTR2 over in OSPF10 / BGP2. Despite receiving 2 LSA's and having the exact same topology / configuration...

I've attached a screen shot of the topology I'm running too to try and aid the tshoot.

Aggregate route:

RTR1-BGP2

show ip bgp

*> 10.125.128.0/17 10.123.255.227 1 32768 ?

RTR2-BGP2

show ip bgp

*> 10.125.128.0/17 10.123.255.227 1 32768 ?

RTR1-OSPF10

show ip route OSPF

O E2    10.125.128.0/17 [110/1] via 10.123.255.227, 00:25:52, FastEthernet2/8

show ip ospf database external 10.125.128.0

OSPF Router with ID (255.255.255.254) (Process ID 10)

Type-5 AS External Link States

Routing Bit Set on this LSA
LS age: 1592
Options: (No TOS-capability, No DC)
LS Type: AS External Link
Link State ID: 10.125.128.0 (External Network Number )
Advertising Router: 10.123.255.226
LS Seq Number: 80000002
Checksum: 0x5F0D
Length: 36
Network Mask: /17
Metric Type: 2 (Larger than any link state path)
TOS: 0
Metric: 1
Forward Address: 10.123.255.227
External Route Tag: 10

Routing Bit Set on this LSA
LS age: 1784
Options: (No TOS-capability, No DC)
LS Type: AS External Link
Link State ID: 10.125.128.0 (External Network Number )
Advertising Router: 10.123.255.227
LS Seq Number: 80000002
Checksum: 0xAA2A
Length: 36
Network Mask: /17
Metric Type: 2 (Larger than any link state path)
TOS: 0
Metric: 1
Forward Address: 0.0.0.0
External Route Tag: 10

RTR2-OSPF10

Show ip route ospf

O E2    10.125.128.0/17 [110/1] via 10.123.255.227, 00:25:11, FastEthernet0/0

show ip ospf database external 10.125.128.0

OSPF Router with ID (255.255.255.255) (Process ID 10)

Type-5 AS External Link States

Routing Bit Set on this LSA
LS age: 1792
Options: (No TOS-capability, No DC)
LS Type: AS External Link
Link State ID: 10.125.128.0 (External Network Number )
Advertising Router: 10.123.255.226
LS Seq Number: 80000002
Checksum: 0x5F0D
Length: 36
Network Mask: /17
Metric Type: 2 (Larger than any link state path)
TOS: 0
Metric: 1
Forward Address: 10.123.255.227
External Route Tag: 10

Routing Bit Set on this LSA
LS age: 166
Options: (No TOS-capability, No DC)
LS Type: AS External Link
Link State ID: 10.125.128.0 (External Network Number )
Advertising Router: 10.123.255.227
LS Seq Number: 80000003
Checksum: 0xA82B
Length: 36
Network Mask: /17
Metric Type: 2 (Larger than any link state path)
TOS: 0
Metric: 1
Forward Address: 0.0.0.0
External Route Tag: 10

None-Aggregate route:

RTR1-BGP2

show ip bgp 

*> 10.125.128.0/19  10.123.255.226           1         32768 ?

RTR2-BGP2

show ip bgp 

*> 10.125.128.0/19  10.123.255.226           1         32768 ?

RTR1-OSPF10

show ip route ospf:

O E2 10.125.128.0/19 [110/1] via 10.123.255.227, 00:06:05, FastEthernet2/8
                                   [110/1] via 10.123.255.226, 00:06:05, FastEthernet2/8

show ip ospf database external 10.125.128.0:

OSPF Router with ID (255.255.255.255) (Process ID 10)

Type-5 AS External Link States

Routing Bit Set on this LSA
LS age: 448
Options: (No TOS-capability, No DC)
LS Type: AS External Link
Link State ID: 10.125.128.0 (External Network Number )
Advertising Router: 10.123.255.226
LS Seq Number: 80000002
Checksum: 0x92E2
Length: 36
Network Mask: /19
Metric Type: 2 (Larger than any link state path)
TOS: 0
Metric: 1
Forward Address: 0.0.0.0
External Route Tag: 10

Routing Bit Set on this LSA
LS age: 448
Options: (No TOS-capability, No DC)
LS Type: AS External Link
Link State ID: 10.125.128.0 (External Network Number )
Advertising Router: 10.123.255.227
LS Seq Number: 80000002
Checksum: 0x8CE7
Length: 36
Network Mask: /19
Metric Type: 2 (Larger than any link state path)
TOS: 0
Metric: 1
Forward Address: 0.0.0.0
External Route Tag: 10

RTR2-OSPF10

show ip route ospf:

O E2 10.125.128.0/19 [110/1] via 10.123.255.227, 00:04:05, FastEthernet0/0
                                   [110/1] via 10.123.255.226, 00:04:05, FastEthernet0/0

show ip ospf database external 10.125.128.0

OSPF Router with ID (255.255.255.254) (Process ID 10)

Type-5 AS External Link States

Routing Bit Set on this LSA
LS age: 565
Options: (No TOS-capability, No DC)
LS Type: AS External Link
Link State ID: 10.125.128.0 (External Network Number )
Advertising Router: 10.123.255.226
LS Seq Number: 80000002
Checksum: 0x92E2
Length: 36
Network Mask: /19
Metric Type: 2 (Larger than any link state path)
TOS: 0
Metric: 1
Forward Address: 0.0.0.0
External Route Tag: 10

Routing Bit Set on this LSA
LS age: 565
Options: (No TOS-capability, No DC)
LS Type: AS External Link
Link State ID: 10.125.128.0 (External Network Number )
Advertising Router: 10.123.255.227
LS Seq Number: 80000002
Checksum: 0x8CE7
Length: 36
Network Mask: /19
Metric Type: 2 (Larger than any link state path)
TOS: 0
Metric: 1
Forward Address: 0.0.0.0
External Route Tag: 10

[Rolf Fischer]

Hi Dan,

the two external LSAs for the summary routes are both E2 with metric 1, which lets them look equal at the first glance.

However, the LSA originated by NXOS-Core-2 has a non-zero forwarding address set:

Advertising Router: 10.123.255.226
Forward Address: 10.123.255.227

Advertising Router: 10.123.255.227
Forward Address: 0.0.0.0

So far I can't see the exact reason for this so I can't say how to avoid it (if possible) yet.

HTH

Rolf

[danielgoing11]

Hi Rolf,

Yes I spotted that, and thanks for supplying the link. 

I'll do some digging to see if I can see why the LSA has a non-zero forwarding address on NXOS-Core-2.

Thanks again,

Dan

[danielgoing11]

Hey Guys,

I fired up the lab yesterday and fixed the routing issues I was seeing in the OSPF process. 

Adding a Layer3 point-to-point link across the vPC for OSPF process 10 and OSPF process 120 corrected the LSA advertisements and FA. 

Despite having layer3 peer-router on the vPC domain, L3 routing information wasn't being shared over the vPC. 

Turns out, it wasn't an issue contained to a summary address, rather affecting all LSA's regardless of summary or not. Only that having the summary and running a fail-over scenario was where I first spotted the loss of ECMP. 

Either, my OSPF topology is wrong for a vPC (as far as I can tell from Brad Hedlund's blog, I've got it right) or Layer 3 over vPC isn't quite working in NX-OS 7.0.3.I5.1. My guess is, the version of code isn't quite there yet for Layer over vPC.

Thanks all for your help and advice - got there in the end.

Thanks

Dan

[danielgoing11 · Accepted solution]

Morning Guys,

I moved my LAB configuration into a live datacentre over the weekend with some interesting results. 

I upgraded the code on a VPC pair of NX-OS 9k's from nxos.7.0.3.I2.3 to nxos.7.0.3.I7.1.

My GNS3 Lab was running VM instances on nxos.7.0.3.I5.1.

Adding layer3 peer-router configuration under the VPC domain on the I7 version of code on the physical hardware didn't show the FA anomaly I was seeing on the OSPF LSA's in the virtual environment running I5. 

To conclude, version I5 of code on the 9k's doesn't quite support Layer3 over VPC in the way it should. I7 seems a more stable version of code for L3 over VPC.

*(On VM instances at least, I've no hardware running I5 and will likely upgrade my remaining 9k's straight to I7)

Thanks all for your help and advice.

Best,

Dan