Transit routing within a VRF is reasonably straight forward, but when I tried to reproduce a problem described in here, I found that need to a little careful to be sure that the routes from one router are passed on to the other.
Here's the toplogy I built
A single tenant has a single VRF and two External Routed Networks (L3Outs) although the same result could have been achieved using a single L3Out. An Access Policy Chain was set up to allow VLAN 2304 to be configured as an SVI on interface Ethernet1/1 of the left ACI Leaf (Leaf 101) and for VLAN 2308 to be configured as an SVI on interface Ethernet1/1 of on the right ACI Leaf (Leaf 102).
First I created a Tenant called Transit with a VRF called DC_VRF
Then I created the first of the two L3Outs. I called it LeftRtr_L3Out and assigned it regular OSPF area 0.0.0.4 and made sure it was linked to the DC_VRF and the appropriate External Routed Domain that held the interface configuation for Ethernet 1/1 and a VLAN pool with VLAN 2304
I give it a Logical Node profile pointing to leaf 101 and an SVI Interface for VLAN 2304 on Ethernet 1/1 on Leaf 101. I used an IP address of 192.168.4.2/24 as I already had a router attached with an IP address of 192.168.4.1/24 on its G0/1.2304 sub-interface.
Next, before OSPF could begin I needed to add a Network, or L3EPG to describe which source IP addresses would be permitted to enter the ACI fabric from the external router. I kept it simple and defined my network as 0.0.0.0/0 - in other words all source IPs were permitted.
At this stage, it is appropriate to look at the external router configuration. The external router was configured with a VRF to contain the configuration of the LeftRouter - here's the VRF config:
ip vrf LeftRouter ! interface GigabitEthernet0/0.2040 encapsulation dot1Q 2040 ip vrf forwarding LeftRouter ip address 192.168.40.1 255.255.255.0 ! interface GigabitEthernet0/1.2304 encapsulation dot1Q 2304 ip vrf forwarding LeftRouter ip address 192.168.4.1 255.255.255.0 ! router ospf 4 vrf LeftRouter router-id 1.1.1.14 passive-interface GigabitEthernet0/0.2040 network 192.168.0.0 0.0.255.255 area 0.0.0.4
And by now the LeftRouter and Leaf101 have become neighbors:
And on the router:
LeftRouter#show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface 1.0.0.4 1 FULL/DR 00:00:38 192.168.4.2 GigabitEthernet0/1.2304
Now things get tricky. Recall from the original diagram that we want Leaf 101 to advertise the 192.168.80.0/24 subnet to the LeftRouter. So we have to tell Leaf101 to advertise this route manually. Here are the steps.
Bingo. Your leaf101 is now ready to advertise the 192.168.80.0/24 subnet - except for one small problem - it hasn't learned about that subnet from the RightRouter yet - so now it's time to "Rinse and Repeat" as they say and repeat the above steps for the Right Router and the route map for the 192.168.40.0/24 subnet.
But of course you won't be able to pass any traffic until there is a contract in place. I simply set one of the L3EPGs to provide the default contract in the common tenant, and the other to consume it.
Finally, some testing from the routers to check that the transit routing is working. First the LeftRouter (note that since I didn't bother advertising the rotues connecting the routers to the fabric, I had to specify the source interface)
LeftRouter#show ip route vrf LeftRouter | include / 1.0.0.0/32 is subnetted, 1 subnets O 1.0.0.4 [110/2] via 192.168.4.2, 02:43:00, GigabitEthernet0/1.2304 192.168.4.0/24 is variably subnetted, 2 subnets, 2 masks C 192.168.4.0/24 is directly connected, GigabitEthernet0/1.2304 L 192.168.4.1/32 is directly connected, GigabitEthernet0/1.2304 192.168.40.0/24 is variably subnetted, 2 subnets, 2 masks C 192.168.40.0/24 is directly connected, GigabitEthernet0/0.2040 L 192.168.40.1/32 is directly connected, GigabitEthernet0/0.2040 O E2 192.168.80.0/24 [110/1] via 192.168.4.2, 00:11:01, GigabitEthernet0/1.2304 LeftRouter#ping vrf LeftRouter 192.168.80.1 source GigabitEthernet0/0.2040 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 192.168.80.1, timeout is 2 seconds: Packet sent with a source address of 192.168.40.1 !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms
Trust me - the RightRouter is the same :) . Here's the config for the Right Router just for completeness sake:
ip vrf RightRouter ! interface GigabitEthernet0/0.2080 encapsulation dot1Q 2080 ip vrf forwarding RightRouter ip address 192.168.80.1 255.255.255.0 ! interface GigabitEthernet0/2.2308 encapsulation dot1Q 2308 ip vrf forwarding RightRouter ip address 192.168.8.1 255.255.255.0 ! router ospf 8 vrf RightRouter router-id 1.1.1.18 passive-interface GigabitEthernet0/0.2080 network 192.168.0.0 0.0.255.255 area 0.0.0.8
And just in case you want to do it yourself, here is the complete Tenant configuation. If you want to copy it to a text file, you can right-click on a tenant, choose Post and select the text file as the source and /uni as the destination.
<?xml version="1.0" encoding="UTF-8"?> <imdata totalCount="1"> <fvTenant annotation="" descr="" dn="uni/tn-Transit" name="Transit" nameAlias="" ownerKey="" ownerTag=""><vnsSvcCont annotation=""/> <rtctrlSubjP annotation="" descr="" name="192.168.40.0:24" nameAlias=""><rtctrlMatchRtDest aggregate="no" annotation="" descr="" ip="192.168.40.0/24" name="" nameAlias=""/></rtctrlSubjP> <rtctrlSubjP annotation="" descr="" name="192.168.80.0:24" nameAlias=""><rtctrlMatchRtDest aggregate="no" annotation="" descr="" ip="192.168.80.0/24" name="" nameAlias=""/></rtctrlSubjP> <l3extOut annotation="" descr="" enforceRtctrl="export" name="RightRtr_L3Out" nameAlias="" ownerKey="" ownerTag="" targetDscp="unspecified"> <rtctrlProfile annotation="" descr="" name="default-export" nameAlias="" ownerKey="" ownerTag="" type="global"> <rtctrlCtxP action="permit" annotation="" descr="" name="192.168.40.0:24_RtCtrlCtx" nameAlias="" order="0"><rtctrlRsCtxPToSubjP annotation="" tnRtctrlSubjPName="192.168.40.0:24"/></rtctrlCtxP> </rtctrlProfile><ospfExtP annotation="" areaCost="1" areaCtrl="redistribute,summary" areaId="0.0.0.8" areaType="regular" descr="" multipodInternal="no" nameAlias=""/><l3extRsL3DomAtt annotation="" tDn="uni/l3dom-T8:WAN-ExtL3Dom"/><l3extRsEctx annotation="" tnFvCtxName="DC_VRF"/> <l3extLNodeP annotation="" configIssues="" descr="" name="Leaf102_NodeProf" nameAlias="" ownerKey="" ownerTag="" tag="yellow-green" targetDscp="unspecified"> <l3extRsNodeL3OutAtt annotation="" configIssues="" rtrId="1.0.0.8" rtrIdLoopBack="yes" tDn="topology/pod-1/node-102"><l3extInfraNodeP annotation="" descr="" fabricExtCtrlPeering="no" fabricExtIntersiteCtrlPeering="no" name="" nameAlias="" spineRole=""/></l3extRsNodeL3OutAtt> <l3extLIfP annotation="" descr="" name="VALN2308_IntProf" nameAlias="" ownerKey="" ownerTag="" tag="yellow-green"> <ospfIfP annotation="" authKeyId="1" authType="none" descr="" name="" nameAlias=""><ospfRsIfPol annotation="" tnOspfIfPolName=""/></ospfIfP><l3extRsPathL3OutAtt addr="192.168.8.2/24" annotation="" autostate="disabled" descr="" encap="vlan-2308" encapScope="local" ifInstT="ext-svi" ipv6Dad="enabled" llAddr="::" mac="00:22:BD:F8:19:FF" mode="regular" mtu="1500" tDn="topology/pod-1/paths-102/pathep-[eth1/1]" targetDscp="unspecified"/><l3extRsNdIfPol annotation="" tnNdIfPolName=""/><l3extRsIngressQosDppPol annotation="" tnQosDppPolName=""/><l3extRsEgressQosDppPol annotation="" tnQosDppPolName=""/></l3extLIfP> </l3extLNodeP> <l3extInstP annotation="" descr="" exceptionTag="" floodOnEncap="disabled" matchT="AtleastOne" name="0.0.0.0:0_L3EPG" nameAlias="" prefGrMemb="exclude" prio="unspecified" targetDscp="unspecified"><l3extSubnet aggregate="" annotation="" descr="" ip="0.0.0.0/0" name="" nameAlias="" scope="import-security"/><fvRsCustQosPol annotation="" tnQosCustomPolName=""/><fvRsCons annotation="" prio="unspecified" tnVzBrCPName="default"/></l3extInstP> </l3extOut> <l3extOut annotation="" descr="" enforceRtctrl="export" name="LeftRtr_L3Out" nameAlias="" ownerKey="" ownerTag="" targetDscp="unspecified"> <rtctrlProfile annotation="" descr="" name="default-export" nameAlias="" ownerKey="" ownerTag="" type="global"> <rtctrlCtxP action="permit" annotation="" descr="" name="192.168.80:24_RtCtrlCtx" nameAlias="" order="0"><rtctrlRsCtxPToSubjP annotation="" tnRtctrlSubjPName="192.168.80.0:24"/></rtctrlCtxP> </rtctrlProfile><ospfExtP annotation="" areaCost="1" areaCtrl="redistribute,summary" areaId="0.0.0.4" areaType="regular" descr="" multipodInternal="no" nameAlias=""/><l3extRsL3DomAtt annotation="" tDn="uni/l3dom-T4:WAN-ExtL3Dom"/><l3extRsEctx annotation="" tnFvCtxName="DC_VRF"/> <l3extLNodeP annotation="" configIssues="" descr="" name="Leaf101_NodeProf" nameAlias="" ownerKey="" ownerTag="" tag="yellow-green" targetDscp="unspecified"> <l3extRsNodeL3OutAtt annotation="" configIssues="" rtrId="1.0.0.4" rtrIdLoopBack="yes" tDn="topology/pod-1/node-101"><l3extInfraNodeP annotation="" descr="" fabricExtCtrlPeering="no" fabricExtIntersiteCtrlPeering="no" name="" nameAlias="" spineRole=""/></l3extRsNodeL3OutAtt> <l3extLIfP annotation="" descr="" name="VLAN2304_IntPorf" nameAlias="" ownerKey="" ownerTag="" tag="yellow-green"> <ospfIfP annotation="" authKeyId="1" authType="none" descr="" name="" nameAlias=""><ospfRsIfPol annotation="" tnOspfIfPolName=""/></ospfIfP><l3extRsPathL3OutAtt addr="192.168.4.2/24" annotation="" autostate="disabled" descr="" encap="vlan-2304" encapScope="local" ifInstT="ext-svi" ipv6Dad="enabled" llAddr="::" mac="00:22:BD:F8:19:FF" mode="regular" mtu="1500" tDn="topology/pod-1/paths-101/pathep-[eth1/1]" targetDscp="unspecified"/><l3extRsNdIfPol annotation="" tnNdIfPolName=""/><l3extRsIngressQosDppPol annotation="" tnQosDppPolName=""/><l3extRsEgressQosDppPol annotation="" tnQosDppPolName=""/></l3extLIfP> </l3extLNodeP> <l3extInstP annotation="" descr="" exceptionTag="" floodOnEncap="disabled" matchT="AtleastOne" name="0.0.0.0:0_L3EPG" nameAlias="" prefGrMemb="exclude" prio="unspecified" targetDscp="unspecified"><fvRsProv annotation="" matchT="AtleastOne" prio="unspecified" tnVzBrCPName="default"/><l3extSubnet aggregate="" annotation="" descr="" ip="0.0.0.0/0" name="" nameAlias="" scope="import-security"/><fvRsCustQosPol annotation="" tnQosCustomPolName=""/></l3extInstP> </l3extOut><fvRsTenantMonPol annotation="" tnMonEPGPolName=""/> <fvCtx annotation="" bdEnforcedEnable="no" descr="" knwMcastAct="permit" name="DC_VRF" nameAlias="" ownerKey="" ownerTag="" pcEnfDir="ingress" pcEnfPref="enforced"><fvRsVrfValidationPol annotation="" tnL3extVrfValidationPolName=""/><vzAny annotation="" descr="" matchT="AtleastOne" name="" nameAlias="" prefGrMemb="disabled"/><fvRsOspfCtxPol annotation="" tnOspfCtxPolName=""/><fvRsCtxToEpRet annotation="" tnFvEpRetPolName=""/><fvRsCtxToExtRouteTagPol annotation="" tnL3extRouteTagPolName=""/><fvRsBgpCtxPol annotation="" tnBgpCtxPolName=""/></fvCtx> </fvTenant> </imdata>
RedNectar
aka Chris Welsh
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: