BD with the same subnet as a l3out not working after upgrade to 5.2.7f

[83881463a]

Hello,

we upgraded our fabric last weekend, everything works fine but we have noticed these days that 4 servers that are in a BD (level3) which has the same subnet definition that an l3out, are not reachable if the traffic flows through that l3out.

We have changed the vlan encapsulation, so, they don't have the same vlan id, the one thing they have in common is the subnet. We can see that the routes are correct and we see the /32 specific routes for the host, in the specific VRF where the l3out is defined.

With the 4.2(7) version, we didn't face this problem, but since the upgrade it stopped working.

Has anyone faced this issue?

Thank you.

Andere

[RedNectar]

Hi @83881463a ,

[Edit 2023.04.01-I thought about this overnight and remembered something that might help]

I remember there being a change in the way subnets are handled in ACI v5.x - but can't find the relevant release note. But to focus on your problem I'd like to know that you have:

• defined each of the 4 servers with /32 IPs under the EPG Subnet
• with the [x] Advertise Externally option selected and
• the [  ] No Default SVI Gateway option unchecked

Then verify of the EXTERNAL router that you are receiving all four /32 routes

RedNectar
aka Chris Welsh

[/Edit: Everything below here is still relevant, but the most relevant part of this answer is above]

---

I started trying to get my head around this, but really need some more specifics; E.g

---

that 4 servers that are in a BD (level3) which has the same subnet definition that an l3out, are not reachable if the traffic flows through that l3out.

OK. How about a diagram? [A picture is worth a thousand words] - oh - and when you paste it, make full page width

the one thing they have in common is the subnet.

OK. Let's see the subnet definition for both

We can see that the routes are correct and we see the /32 specific routes for the host, in the specific VRF where the l3out is defined.

Well. ACI is not that simple - EVERY switch has a different view of the same VRF - so make sure you include the output of a show ip route vrf <tenant>:<vrf> on (a) the switch(es) where the servers are and (b) the switch(es) where the router(s) are

---

Tip: You can issue a command on any switch from the APIC using the fabric <swith-id>  format of the command. E.g - if the relevant switch IDs are 101, 103 and 104:

fabric 101,103-104 show ip route vrf <tenant>:<vrf> 

would issue the same command on all three switches without having to ssh to each switch.

---

[83881463a]

Hello,

it seems that the TAC have found a bug related to this. I am going to collect some logs tomorrow afternoon, I will tell you what they say to me.

In the mean time, I explain the situation a little bit and the workaround, we luckily found.

The diagram, is like this (I have a simplify it a little it):

In the LEAFs, 101 and 102, the l3out is configured to the router and in the vrf OFFICES I can see the correct routes towards the 4 servers in the LEAFs 101, 102, 103 and 104, that were established in the LEAFs 103 y 104. In the previous version, everything worked fine.

We needed to establish the same routes in the LEAFs 101 y 102 for this to work, as a work around.

Regards.

Andere

[RedNectar]

Andre (@83881463a ),

Thanks for the update and the diagram - which makes it much clearer. If you find out the bugID I'd like to know.

[RedNectar]

Andere ( @83881463a ) - do you need more help with this?

If your question has been answered, it is a great idea to mark the question as being answered.  This helps:

1. others with a similar problem find the correct answer
2. people who look for "unanswered" questions to answer finding this
3. prevent your question from becoming a "dead thread"