we upgraded our fabric last weekend, everything works fine but we have noticed these days that 4 servers that are in a BD (level3) which has the same subnet definition that an l3out, are not reachable if the traffic flows through that l3out.
We have changed the vlan encapsulation, so, they don't have the same vlan id, the one thing they have in common is the subnet. We can see that the routes are correct and we see the /32 specific routes for the host, in the specific VRF where the l3out is defined.
With the 4.2(7) version, we didn't face this problem, but since the upgrade it stopped working.
[/Edit: Everything below here is still relevant, but the most relevant part of this answer is above]
I started trying to get my head around this, but really need some more specifics; E.g
that 4 servers that are in a BD (level3) which has the same subnet definition that an l3out, are not reachable if the traffic flows through that l3out.
OK. How about a diagram? [A picture is worth a thousand words] - oh - and when you paste it, make full page width
the one thing they have in common is the subnet.
OK. Let's see the subnet definition for both
We can see that the routes are correct and we see the /32 specific routes for the host, in the specific VRF where the l3out is defined.
Well. ACI is not that simple - EVERY switch has a different view of the same VRF - so make sure you include the output of a show ip route vrf <tenant>:<vrf> on (a) the switch(es) where the servers are and (b) the switch(es) where the router(s) are
Tip: You can issue a command on any switch from the APIC using the fabric <swith-id> format of the command. E.g - if the relevant switch IDs are 101, 103 and 104:
fabric 101,103-104 show ip route vrf <tenant>:<vrf>
would issue the same command on all three switches without having to ssh to each switch.
RedNectar aka Chris Welsh. Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.
it seems that the TAC have found a bug related to this. I am going to collect some logs tomorrow afternoon, I will tell you what they say to me.
In the mean time, I explain the situation a little bit and the workaround, we luckily found.
The diagram, is like this (I have a simplify it a little it):
In the LEAFs, 101 and 102, the l3out is configured to the router and in the vrf OFFICES I can see the correct routes towards the 4 servers in the LEAFs 101, 102, 103 and 104, that were established in the LEAFs 103 y 104. In the previous version, everything worked fine.
We needed to establish the same routes in the LEAFs 101 y 102 for this to work, as a work around.