cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
3890
Views
20
Helpful
3
Comments
RedNectar
VIP
VIP

Transit routing within a VRF is reasonably straight forward, but when I tried to reproduce a problem described in here, I found that need to a little careful to be sure that the routes from one router are passed on to the other.

Here's the toplogy I built

Topology.png

 

A single tenant has a single VRF and two External Routed Networks (L3Outs) although the same result could have been achieved using a single L3Out. An Access Policy Chain was set up to allow VLAN 2304 to be configured as an SVI on interface Ethernet1/1 of the left ACI Leaf (Leaf 101) and for VLAN 2308 to be configured as an SVI on interface Ethernet1/1 of on the right ACI Leaf (Leaf 102).

First I created a Tenant called Transit with a VRF called DC_VRF

DC_VRF.png

Then I created the first of the two L3Outs. I called it LeftRtr_L3Out and assigned it regular OSPF area 0.0.0.4 and made sure it was linked to the DC_VRF and the appropriate External Routed Domain that held the interface configuation for Ethernet 1/1 and a VLAN pool with VLAN 2304

LeftRouter_L3OutOSPF.png

I give it a Logical Node profile pointing to leaf 101 and an SVI Interface for VLAN 2304 on Ethernet 1/1 on Leaf 101.  I used an IP address of 192.168.4.2/24 as I already had a router attached with an IP address of 192.168.4.1/24 on its G0/1.2304 sub-interface.

LeftRouter_L3Out.png

Next, before OSPF could begin I needed to add a Network, or L3EPG to describe which source IP addresses would be permitted to enter the ACI fabric from the external router. I kept it simple and defined my network as 0.0.0.0/0 - in other words all source IPs were permitted.

0.0.0.0_L3EPG.png

At this stage, it is appropriate to look at the external router configuration. The external router was configured with a VRF to contain the configuration of the LeftRouter - here's the VRF config:

ip vrf LeftRouter
!
interface GigabitEthernet0/0.2040
 encapsulation dot1Q 2040
 ip vrf forwarding LeftRouter
 ip address 192.168.40.1 255.255.255.0
!
interface GigabitEthernet0/1.2304
 encapsulation dot1Q 2304
 ip vrf forwarding LeftRouter
 ip address 192.168.4.1 255.255.255.0
!
 router ospf 4 vrf LeftRouter
 router-id 1.1.1.14
 passive-interface GigabitEthernet0/0.2040
 network 192.168.0.0 0.0.255.255 area 0.0.0.4

And by now the LeftRouter and Leaf101 have become neighbors:

OSPFNeighbors.png

 And on the router:

LeftRouter#show ip ospf neighbor
Neighbor ID     Pri   State           Dead Time   Address         Interface
1.0.0.4           1   FULL/DR         00:00:38    192.168.4.2     GigabitEthernet0/1.2304

Now things get tricky.  Recall from the original diagram that we want Leaf 101 to advertise the 192.168.80.0/24 subnet to the LeftRouter. So we have to tell Leaf101 to advertise this route manually.  Here are the steps.

  1. Create a Match Rule for Route Maps (in your Tenant, navigate to Networking > External Routed Networks >+ Create Match Rule for a Route Map)
    1. I Called mine192.168.80:24 and added a Match Prefix of 192.168.80.0/24
      MatchRule.png
  2. Now to tell Leaf101 you want to advertise this prefix, configure the default-export route map and set it to Match Routing Policy Only, and link it to the Match Rule you created earlier. (in your Tenant, navigate to Networking > External Routed Networks > LeftRtr_L3Out > Route Maps/Profiles >+ Create Route Map/Profile)
    1. Name the profile default-export (or pick it from the drop down list) and set the Type to Match Routing Policy Only
    2. Add a context - I called mine 192.168.80.0:24_RtCtrlCtx - and choose the match rule (192.168.80:24 in my case) you created earlier.
      RouteControlContext.png

Bingo. Your leaf101 is now ready to advertise the 192.168.80.0/24 subnet - except for one small problem - it hasn't learned about that subnet from the RightRouter yet - so now it's time to "Rinse and Repeat" as they say and repeat the above steps for the Right Router and the route map for the 192.168.40.0/24 subnet.

But of course you won't be able to pass any traffic until there is a contract in place.  I simply set one of the L3EPGs to provide the default contract in the common tenant, and the other to consume it.

defaultContract.png

Finally, some testing from the routers to check that the transit routing is working. First the LeftRouter (note that since I didn't bother advertising the rotues connecting the routers to the fabric, I had to specify the source interface)

LeftRouter#show ip route vrf LeftRouter | include /
      1.0.0.0/32 is subnetted, 1 subnets
O        1.0.0.4 [110/2] via 192.168.4.2, 02:43:00, GigabitEthernet0/1.2304
      192.168.4.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.4.0/24 is directly connected, GigabitEthernet0/1.2304
L        192.168.4.1/32 is directly connected, GigabitEthernet0/1.2304
      192.168.40.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.40.0/24 is directly connected, GigabitEthernet0/0.2040
L        192.168.40.1/32 is directly connected, GigabitEthernet0/0.2040
O E2  192.168.80.0/24
           [110/1] via 192.168.4.2, 00:11:01, GigabitEthernet0/1.2304
LeftRouter#ping vrf LeftRouter 192.168.80.1 source  GigabitEthernet0/0.2040
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.80.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.40.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

Trust me - the RightRouter is the same :) . Here's the config for the Right Router just for completeness sake:

ip vrf RightRouter
!
interface GigabitEthernet0/0.2080
 encapsulation dot1Q 2080
 ip vrf forwarding RightRouter
 ip address 192.168.80.1 255.255.255.0
!
interface GigabitEthernet0/2.2308
 encapsulation dot1Q 2308
 ip vrf forwarding RightRouter
 ip address 192.168.8.1 255.255.255.0
!
router ospf 8 vrf RightRouter
 router-id 1.1.1.18
 passive-interface GigabitEthernet0/0.2080
 network 192.168.0.0 0.0.255.255 area 0.0.0.8

And just in case you want to do it yourself, here is the complete Tenant configuation. If you want to copy it to a text file, you can right-click on a tenant, choose Post and select the text file as the source and /uni as the destination.

 

<?xml version="1.0" encoding="UTF-8"?>
<imdata totalCount="1">
  <fvTenant annotation="" descr="" dn="uni/tn-Transit" name="Transit" nameAlias="" ownerKey="" ownerTag=""><vnsSvcCont annotation=""/>
    <rtctrlSubjP annotation="" descr="" name="192.168.40.0:24" nameAlias=""><rtctrlMatchRtDest aggregate="no" annotation="" descr="" ip="192.168.40.0/24" name="" nameAlias=""/></rtctrlSubjP>
    <rtctrlSubjP annotation="" descr="" name="192.168.80.0:24" nameAlias=""><rtctrlMatchRtDest aggregate="no" annotation="" descr="" ip="192.168.80.0/24" name="" nameAlias=""/></rtctrlSubjP>
    <l3extOut annotation="" descr="" enforceRtctrl="export" name="RightRtr_L3Out" nameAlias="" ownerKey="" ownerTag="" targetDscp="unspecified">
      <rtctrlProfile annotation="" descr="" name="default-export" nameAlias="" ownerKey="" ownerTag="" type="global">
        <rtctrlCtxP action="permit" annotation="" descr="" name="192.168.40.0:24_RtCtrlCtx" nameAlias="" order="0"><rtctrlRsCtxPToSubjP annotation="" tnRtctrlSubjPName="192.168.40.0:24"/></rtctrlCtxP>
      </rtctrlProfile><ospfExtP annotation="" areaCost="1" areaCtrl="redistribute,summary" areaId="0.0.0.8" areaType="regular" descr="" multipodInternal="no" nameAlias=""/><l3extRsL3DomAtt annotation="" tDn="uni/l3dom-T8:WAN-ExtL3Dom"/><l3extRsEctx annotation="" tnFvCtxName="DC_VRF"/>
      <l3extLNodeP annotation="" configIssues="" descr="" name="Leaf102_NodeProf" nameAlias="" ownerKey="" ownerTag="" tag="yellow-green" targetDscp="unspecified">
        <l3extRsNodeL3OutAtt annotation="" configIssues="" rtrId="1.0.0.8" rtrIdLoopBack="yes" tDn="topology/pod-1/node-102"><l3extInfraNodeP annotation="" descr="" fabricExtCtrlPeering="no" fabricExtIntersiteCtrlPeering="no" name="" nameAlias="" spineRole=""/></l3extRsNodeL3OutAtt>
        <l3extLIfP annotation="" descr="" name="VALN2308_IntProf" nameAlias="" ownerKey="" ownerTag="" tag="yellow-green">
          <ospfIfP annotation="" authKeyId="1" authType="none" descr="" name="" nameAlias=""><ospfRsIfPol annotation="" tnOspfIfPolName=""/></ospfIfP><l3extRsPathL3OutAtt
            addr="192.168.8.2/24"
            annotation=""
            autostate="disabled"
            descr=""
            encap="vlan-2308"
            encapScope="local"
            ifInstT="ext-svi"
            ipv6Dad="enabled"
            llAddr="::"
            mac="00:22:BD:F8:19:FF"
            mode="regular"
            mtu="1500"
            tDn="topology/pod-1/paths-102/pathep-[eth1/1]"
            targetDscp="unspecified"/><l3extRsNdIfPol annotation="" tnNdIfPolName=""/><l3extRsIngressQosDppPol annotation="" tnQosDppPolName=""/><l3extRsEgressQosDppPol annotation="" tnQosDppPolName=""/></l3extLIfP>
      </l3extLNodeP>
      <l3extInstP annotation="" descr="" exceptionTag="" floodOnEncap="disabled" matchT="AtleastOne" name="0.0.0.0:0_L3EPG" nameAlias="" prefGrMemb="exclude" prio="unspecified" targetDscp="unspecified"><l3extSubnet aggregate="" annotation="" descr="" ip="0.0.0.0/0" name="" nameAlias="" scope="import-security"/><fvRsCustQosPol annotation="" tnQosCustomPolName=""/><fvRsCons annotation="" prio="unspecified" tnVzBrCPName="default"/></l3extInstP>
    </l3extOut>
    <l3extOut annotation="" descr="" enforceRtctrl="export" name="LeftRtr_L3Out" nameAlias="" ownerKey="" ownerTag="" targetDscp="unspecified">
      <rtctrlProfile annotation="" descr="" name="default-export" nameAlias="" ownerKey="" ownerTag="" type="global">
        <rtctrlCtxP action="permit" annotation="" descr="" name="192.168.80:24_RtCtrlCtx" nameAlias="" order="0"><rtctrlRsCtxPToSubjP annotation="" tnRtctrlSubjPName="192.168.80.0:24"/></rtctrlCtxP>
      </rtctrlProfile><ospfExtP annotation="" areaCost="1" areaCtrl="redistribute,summary" areaId="0.0.0.4" areaType="regular" descr="" multipodInternal="no" nameAlias=""/><l3extRsL3DomAtt annotation="" tDn="uni/l3dom-T4:WAN-ExtL3Dom"/><l3extRsEctx annotation="" tnFvCtxName="DC_VRF"/>
      <l3extLNodeP annotation="" configIssues="" descr="" name="Leaf101_NodeProf" nameAlias="" ownerKey="" ownerTag="" tag="yellow-green" targetDscp="unspecified">
        <l3extRsNodeL3OutAtt annotation="" configIssues="" rtrId="1.0.0.4" rtrIdLoopBack="yes" tDn="topology/pod-1/node-101"><l3extInfraNodeP annotation="" descr="" fabricExtCtrlPeering="no" fabricExtIntersiteCtrlPeering="no" name="" nameAlias="" spineRole=""/></l3extRsNodeL3OutAtt>
        <l3extLIfP annotation="" descr="" name="VLAN2304_IntPorf" nameAlias="" ownerKey="" ownerTag="" tag="yellow-green">
          <ospfIfP annotation="" authKeyId="1" authType="none" descr="" name="" nameAlias=""><ospfRsIfPol annotation="" tnOspfIfPolName=""/></ospfIfP><l3extRsPathL3OutAtt
            addr="192.168.4.2/24"
            annotation=""
            autostate="disabled"
            descr=""
            encap="vlan-2304"
            encapScope="local"
            ifInstT="ext-svi"
            ipv6Dad="enabled"
            llAddr="::"
            mac="00:22:BD:F8:19:FF"
            mode="regular"
            mtu="1500"
            tDn="topology/pod-1/paths-101/pathep-[eth1/1]"
            targetDscp="unspecified"/><l3extRsNdIfPol annotation="" tnNdIfPolName=""/><l3extRsIngressQosDppPol annotation="" tnQosDppPolName=""/><l3extRsEgressQosDppPol annotation="" tnQosDppPolName=""/></l3extLIfP>
      </l3extLNodeP>
      <l3extInstP annotation="" descr="" exceptionTag="" floodOnEncap="disabled" matchT="AtleastOne" name="0.0.0.0:0_L3EPG" nameAlias="" prefGrMemb="exclude" prio="unspecified" targetDscp="unspecified"><fvRsProv annotation="" matchT="AtleastOne" prio="unspecified" tnVzBrCPName="default"/><l3extSubnet aggregate="" annotation="" descr="" ip="0.0.0.0/0" name="" nameAlias="" scope="import-security"/><fvRsCustQosPol annotation="" tnQosCustomPolName=""/></l3extInstP>
    </l3extOut><fvRsTenantMonPol annotation="" tnMonEPGPolName=""/>
    <fvCtx annotation="" bdEnforcedEnable="no" descr="" knwMcastAct="permit" name="DC_VRF" nameAlias="" ownerKey="" ownerTag="" pcEnfDir="ingress" pcEnfPref="enforced"><fvRsVrfValidationPol annotation="" tnL3extVrfValidationPolName=""/><vzAny annotation="" descr="" matchT="AtleastOne" name="" nameAlias="" prefGrMemb="disabled"/><fvRsOspfCtxPol annotation="" tnOspfCtxPolName=""/><fvRsCtxToEpRet annotation="" tnFvEpRetPolName=""/><fvRsCtxToExtRouteTagPol annotation="" tnL3extRouteTagPolName=""/><fvRsBgpCtxPol annotation="" tnBgpCtxPolName=""/></fvCtx>
  </fvTenant>
</imdata>

 

RedNectar
aka Chris Welsh

 

 

3 Comments
leo.espinosa
Level 1
Level 1

Hi RedNetctar,

thanks for the document.

i have kind of the same deployment but using bgp on top. i'm not able to pass traffic even when everything is configured kind of same (leaf reachability both sides+contract) and i see each other routes in both ends.

Do you have any advice about troubleshooting this situation in ACI?

Thanks,

Leo

RedNectar
VIP
VIP

Hi Leo,

Unfortunately I don't have a solution right away, I'd have to build it to check, and won't have the time to do so for a while. If you haven't figured it out by next weekend, send me a PM and I'll see if I can get time to build it next weekend.

 

leo.espinosa
Level 1
Level 1

Hi RedNectar,

i actually solved the issue. btw, it was related to BGP config (inside ACI), i forgot to configure the next-hop-self option, it is key in BGP transit routing using different leafs, which took me more time than expected to find. It was helpfull to connect into the leaf console to check the bgp routing tables.

Thanks!

Leo

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: