Issue: Nexus 9000 VXLAN border VTEP vrf route-leaking a default route to other VTEPs

[ledzepp817]

Hello all,

I'm trying to have route-leak a default route from a Shared VRF to a Customer VRF.  Very similar to this document 

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/92x/vxlan-92x/configuration/guide/b-cisco-nexus-9000-series-nx-os-vxlan-configuration-guide-92x/b_Cisco_Nexus_9000_Series_NX-OS_VXLAN_Configuration_Guide_9x_chapter_0101.html

(particularly the second to last example where you have a shared VRF and are leaking default to another VRF on a border node):

I have a similar design, a border VXLAN node with 2 VRFs,  one shared VRF has a default route out to Internet and will leak the default route to the other Customer VRF.   Then all downstream VXLAN nodes should get this default route in the Customer VRF (similar to the "Red" or the "Blue" one in the document).

The configuration example has this working except you are supposed to configure "ip route 0.0.0.0/0 Null0" under the Customer VRF context.  This route overrides the imported 0.0.0.0/0 route from the shared VRF so all traffic just gets routed to Null0 which isn't what I want (or would anyone want?).  This route propogates to all other VXLAN nodes so you see a default route in BGP/the route table headed for the border node, but like the issue above, all traffic for the Customer VRF dies at the border node since its routed to Null0.

If I remove this Null0 route, then the Customer VRF has the correct default route to the shared VRF on the border node, but this route isn't propagated to any other VXLAN nodes for that Customer VRF (maybe something about no exporting any imported routes for protecting routing loops, not sure if its the same here).  

If I try to hack the static Null0 route with a 250 AD, then its the same behavior as if I don't have the Null0 route (the border node has correct default route for Customer VRF, but its not propogated).  its like the border node Customer VRF will not advertise the 0.0.0.0/0 route if the default route in its routing table is the leaked one.

Any ideas?  Thanks to everyone.

[f00z]

I went through all this a while ago and had the same result. I have a thread on here where I had a large number of problems getting very simplistic things to work on the 9k hardware.

Bottom line is you need to have the route in BGP in the internet vrf and import it into the customer vrf. you need every single customer vrf to be configured on the border node with at least one route imported (which is ridiculous) or it will block sending to the customer vrf (some sort of auto-acl) , you also need to import the default route on every leaf (as it won't export routes previously imported) AND have the internet vrf on every leaf with the default route (or at least one route imported).

The two ways around this don't work on the hardware (when you actually test it on a switch) but they work in the software, which was really REALLY frustrating and got me so burned out on it i stopped working with it.

One way is to allow traffic to send to a VNI that doesn't exist locally , which works in the software testing but when u try it on actual hardware it fails.

Another way is to re-export an already imported route which breaks the rule of not exporting previously imported routes (which apparently is a way to do but only on the newer platforms? which makes no sense because it's a pure software thing). 

[ledzepp817]

Thank you friend.  I was able to get what I wanted but you are right, I had to export/import the routes at every node which doesn't really do what I want.  Very strange.   Looks like I'll have to engineer a different design.