Bridge Domain Subnets as Static in Border Lea

[Anuj Singhi]

Hi Members,

I have a L3 out for backup service between fabric and ASR over which ospf is running. Now when I tried to trace down the subnet backwards from the ASR, it took me to the L3 Border leaf which is right. Then I ran the "show ip route vrf xx z.z.z.z" command but it gave me below result:

Leaf-1 show ip route vrf IT:CTX-IT-Backup 10.10.10.192
IP Route Table for VRF "ITcorp:CTX-IT-BACKUP"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

10.10.10.192/29, ubest/mbest: 1/0, attached, direct, pervasive
*via 10.20.20.65%overlay-1, [1/0], 28w199d, static
recursive next hop: 10.20.20.65/32%overlay-1

10.20.20.65 is the vtep ip of spine. I thought it would give me the vtep ip of the leaf where the EPG corresponding to BD is deployed. When I checked in spine it gave me the vtep ip of both border leafs.

I randomly checked few more bridge domain subnets and found out that all of them are learned as static in border leafs. I am really having difficulty in understanding why the subnets are being learned as staic and not through ospf  as the l3out is attached to the BD's.

regards,

Anuj

[RedNectar]

Hi Anuj,

It would seem that in your case the 10.10.10.192/29 subnet is the subnet of the Bridge Domain in question - you did not make that clear in your question.

Assuming this is the case, then you need to understand the concept of the Pervasive Gateway or Virtual Default Gateway 

The idea is that when you define an IP address on a BD or EPG, it gets to exist on every Leaf that has an endpoint for the BD/EPG attached.  Hence you see the following in your routing table for any leaf that has an attached endpoint for the BD/EPG.

10.10.10.192/29, ubest/mbest: 1/0, attached, direct, pervasive

Now, the BD is likely to actually exists on several leaves, so your routing table needs to know how to get to those other leaves, hence the second entry in your routing table.

*via 10.20.20.65%overlay-1, [1/0], 28w199d, static

which of course indicates that to get to any other leaf, it needs to get there via the spine.

I hope this helps

---

Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

---

[Anuj Singhi]

Hi Nectar,

"It would seem that in your case the 10.10.10.192/29 subnet is the subnet of the Bridge Domain in question"- Yes that is what I meant and yes I understood the first entry related to pervasive gateway and how does it work but my doubt is somewhat different.

Firstly - I guess the first entry will only appear if that EPG is binded to any port in the respective switch which in my case is "LEAF-1", but here the epg has no local port in this leaf and the route is present because of the L3-OUT that I have associated with the BD and the leaf being the border leaf which is having ospf running with external router. So why is it still showing "pervasive"

Secondly- I got the point that it will show spine as your next hop but lets say that the EPG is binded to one port in leaf 3 (vtep-10.20.20.20), then why isn't showing next hop is the other leaf-3 vtep ip as connected subnet(because it will be local to leaf-3). and why its showing as "STATIC" thats my big question.

Regards,

Anuj

[Anuj Singhi]

@RedNectar appreciate your inputs to my last query.

[micgarc2 · Accepted solution]

When you define a BD subnet it is a pushed as a static pervasive route on the leaf (which you see on the the <show ip route vrf x> output). That static route is then put into an export static route map and then redistributed into whatever protocol (OSPF for ex) the L3 out is you tied to the the BD is. If you do a <show ip ospf route vrf x> on the leaf you should see the route as a type-2 route. From the perspective of the the external device you will see this BD route was advertised as an external type-2 route from the leaf.

Ex:

BD SVI: 10.10.10.254/24  /  L3 is OSPF

Leaf:

a-leaf101# show ip route vrf MG:v1
IP Route Table for VRF "MG:v1"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

1.1.1.101/32, ubest/mbest: 2/0, attached, direct
*via 1.1.1.101, lo11, [1/0], 6d22h, local, local
*via 1.1.1.101, lo11, [1/0], 6d22h, direct
1.1.1.103/32, ubest/mbest: 1/0
*via 10.0.72.64%overlay-1, [1/0], 18:13:21, bgp-64516, internal, tag 64516
10.10.10.0/24, ubest/mbest: 1/0, attached, direct, pervasive
*via 10.0.120.34%overlay-1, [1/0], 18:13:21, static
10.10.10.254/32, ubest/mbest: 1/0, attached, pervasive
*via 10.10.10.254, vlan104, [1/0], 6d22h, local, local

a-leaf101# show ip ospf route vrf MG:v1
OSPF Process ID default VRF MG:v1, Routing Table
(D) denotes route is directly attached (R) denotes route is in RIB
1.1.1.101/32 (intra)(D) area backbone
via 1.1.1.101/lo11* , cost 1 distance 110
1.1.1.103/32 (intra)(R) area backbone
via 192.168.1.2/Vlan185 , cost 5 distance 110
10.10.10.0/24 (type-2)(R) area backbone
via 192.168.1.2/Vlan185 , cost 20 distance 110
100.100.100.0/24 (intra)(R) area backbone
via 192.168.1.3/Vlan185 , cost 5 distance 110
192.168.1.0/24 (intra)(D) area backbone
via 192.168.1.1/Vlan185* , cost 4 distance 110
a-leaf101# 

a-leaf101# show ip ospf vrf MG:v1

Routing Process default with ID 1.1.1.101 VRF MG:v1
Stateful High Availability enabled
Supports only single TOS(TOS0) routes
Supports opaque LSA
Table-map using route-map exp-ctx-2457601-deny-external-tag
Redistributing External Routes from
static route-map exp-ctx-st-2457601
direct route-map exp-ctx-st-2457601
eigrp route-map exp-ctx-proto-2457601
bgp route-map exp-ctx-proto-2457601
Maximum number of non self-generated LSA allowed 20000
(feature configured but inactive)
Current number of non self-generated LSA 3
Threshold for warning message 75%
Administrative distance 110
Reference Bandwidth is 40000 Mbps
SPF throttling delay time of 200.000 msecs,
SPF throttling hold time of 1000.000 msecs,
SPF throttling maximum wait time of 5000.000 msecs
LSA throttling start time of 0.000 msecs,
LSA throttling hold interval of 5000.000 msecs,
LSA throttling maximum wait time of 5000.000 msecs
Minimum LSA arrival 1000.000 msec
LSA group pacing timer 10 secs
Maximum paths to destination 8
Number of external LSAs 2, checksum sum 0x0
Number of opaque AS LSAs 0, checksum sum 0x0
Number of areas is 1, 1 normal, 0 stub, 0 nssa
Number of active areas is 1, 1 normal, 0 stub, 0 nssa
Area (backbone)
Area has existed for 6d23h
Interfaces in this area: 2 Active interfaces: 2
Passive interfaces: 1 Loopback interfaces: 1
SPF calculation has run 31 times
Last SPF ran for 0.002406s
Area ranges are
Area-filter in 'exp-ctx-proto-2457601'
Number of LSAs: 4, checksum sum 0x0

a-leaf101# show route-map exp-ctx-st-2457601
route-map exp-ctx-st-2457601, deny, sequence 1
Match clauses:
tag: 4294967295
Set clauses:
route-map exp-ctx-st-2457601, permit, sequence 15801
Match clauses:
ip address prefix-lists: IPv4-st32772-2457601-exc-int-inferred-export-dst
ipv6 address prefix-lists: IPv6-deny-all
Set clauses:

sa-leaf101# show ip prefix-list IPv4-st32772-2457601-exc-int-inferred-export-dst
ip prefix-list IPv4-st32772-2457601-exc-int-inferred-export-dst: 1 entries
seq 1 permit 10.10.10.254/24

External Device:

ACI-5596-A# show ip ospf route vrf mg1
OSPF Process ID 2 VRF mg1, Routing Table
(D) denotes route is directly attached (R) denotes route is in RIB
1.1.1.101/32 (intra)(R) area 0.0.0.0
via 192.168.1.1/Vlan1300 , cost 41 distance 110
1.1.1.103/32 (intra)(R) area 0.0.0.0
via 192.168.1.2/Vlan1300 , cost 41 distance 110
10.10.10.0/24 (type-2)(R) area 0.0.0.0
via 192.168.1.1/Vlan1300 , cost 20 distance 110
via 192.168.1.2/Vlan1300 , cost 20 distance 110
100.100.100.0/24 (intra)(D) area 0.0.0.0
via 100.100.100.0/Lo50* , cost 1 distance 110
192.168.1.0/24 (intra)(D) area 0.0.0.0
via 192.168.1.3/Vlan1300* , cost 40 distance 110

Thank you for participating in the Cisco Support Forum for ACI! If you have other questions related to this post, please let us know. If this response answers your questions, please mark this post "answered" and assign a rating to the response(s) provided. This will help notify other viewers that your question(s) is answered and this helps us provide better responses for this and future questions.

Regards,

Michael G.

[Anuj Singhi]

@micgarc2 thanks for the awesome explanation but when I follow the same step in my border leaf I didnt get the subnet in "show ip ospf route vrf X". I think our commands are on the leaf where the epg corresponding to the BD is binding to some interface.

In my border leaf, how do I get to the leaf where this subnet(epg has static binding) is defined ? when I did show ip route vrf x in border leaf, it gave me vtep ip of spine and when I ran the same command in spine it threw my back to the border leaf.