11-28-2019 12:00 AM
Hi,
I've successfully configured the inband access in ACI using the mgmt tenant with the inb VRF. Consuming the L3out from the common tenant, I can remotely manage the APIC and switches, however, I cannot connect from another EPG in another tenant. The inband subnet and the consuming EPG subnet are both configured for "Shared between VRFs" and correctly leaked at each VRF after checking the routes. I've exported the inband provider contract and added it into the consumer EPG as a contract interface, but still cannot reach the inband IPs. Is this a known ACI limitation?
Regards.
Solved! Go to Solution.
11-30-2019 07:47 PM - edited 11-30-2019 10:38 PM
Hi @Antonio Macia ,
OK, I worked through it (making a couple of my own mistakes on the way) and got it working. I documented the complete process on my blog, but carrying on from:
a simple contract that allows SSH only:
This contract is provided by the inband EPG at the "mgmt" Tenant and exported to tenant B. EPG at Tenant B consumes the contract interface. Can't SSH the APICs or switches from a VM in Tenant B.
The bit you were missing is actually not so much what YOU were missing, but what ACI is missing, and that is the ablity to mark the inband EPG subnet as Shared Between VRFs.
The kludge I used to get around this was to create an Application Profile and EPG in the mgmt tenant, added the inband management IP subnet and checked the shared between VRFs option, and had it also provide the contract.
That allowed the route for the inband management subnet to leak into Tenant B, and it all came good.
Don't forget to check out the complete story.
I hope this helps
Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem
Footnote: Regarding the inability to set the Shared between VRFs option on the Inband EPG - I tried forcing the attribute by posting this to the EPG:
{ "fvSubnet": { "attributes": { "annotation": "", "ctrl": "nd", "descr": "", "ip": "192.168.99.1/24", "name": "", "nameAlias": "", "preferred": "no", "scope": "private,shared", "virtual": "no" } } }
into the mgmt tenant - the post worked, and I could see the changed attribute in the object nrowser, but it made no difference to the behaviour. So to fix this properly needs more than just adding the abilty to set the option.
12-01-2019 10:28 AM - edited 12-01-2019 10:42 AM
There is a simpler way to let EPG in another TENANT:VRF to get to inb. The issue is a route leak from inb to the client's VRF that needs an EPG subnet, not just a BD subnet. That is a bit different to config. in MGMT / INB epg compared to the "normal" EPG.
So, without dealing with EPG subnet: create global contract for ICMP in MGMP tenant apply it as provider to INB EPG and export to client's tenant and apply contract interface to clients VRF. Cleate . contract for ICMP in client's tenant, apply as prover to client's EPG and export it to MGMT and apply contact interface to INB EPG. That will take care or route leak both directions.
Also, i it is nothing wrong, i believe, to allow ping between client's IPs and inb IPs. Could be helpful in future.
Now you can add your SSH contact as client's EPG consumer and INB EPG provider.
If you really want to make SSH one way only, in subject un-check "reversed port" and "both direction" and make 2 rules in the subject:
Consumer-provider source TCP (default) 1024-65535 to TCP 22
Provider Consumer TCP (stateful, Established ) TCP/22 to TCP/1025-65535
11-28-2019 05:52 AM - edited 11-28-2019 05:53 AM
I know it is obvious, but did you check route in VRF mgmt:inb for routes back to your consumers ?
11-28-2019 05:58 AM
Hi,
Yes, the consumer routes are present in the "inb" VRF, so routing should happen normally.
11-28-2019 06:32 AM
If it's not a route, it is contract. Is you contract subject has specific ports, by directional ? Do you have ICMP permitted ? Can you ping ?
The configuration should work, right now , i'm accessing APIC inb from the workstation in another tenant ...
11-28-2019 07:14 AM
How are your consumer and provider contracts configured?
I think I've tried almost every possible option for the provider contract (always allowing all kind of traffic):
Do you reach the mgmt tenant through an external L3out or within the ACI fabric?
11-28-2019 08:31 AM
Contract should be "global" . I have access to inb form both, L3out in number of tenants and EPGs in number of tenants.
What about subject in the contract in the subject? Do did you try unrestricted: IP /both dir./ reverse port
11-28-2019 11:51 PM
My contract is set to global and the filter allows all the traffic.
Surprisingly, if I configure the consumer EPG as the "provider" and vice-versa (the other way around), I can reach the inband EPG. For example, when the supposed consumer EPG provides ICMP access and this service is consumed by the inband EPG, then I can ping the APIC and switches, the same when the filter allows SSH (source port 22) at the consumer, then I can ssh the inband IPs. So it's like the contracts, when provided by the inband EPG, do not work for some reason.
11-29-2019 12:35 AM
Looks like you answered my question while I was composing it........ Do you have psychic power?
"the filter allows all the traffic."
This means the concept of Provider and Consumer go right out the window - in fact the Cosumer/Provider ONLY works for TCP. For ICMP, UDP, ESP etc etc, it makes no difference which is provider and which is consumer.
Hence your L3Out EPGs and access inside EPGs and vice versa
I hope this helps
Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem
11-29-2019 01:52 AM - edited 11-29-2019 02:55 AM
Hi @RedNectar ,
Right now I have a simple contract that allows SSH only:
This contract is provided by the inband EPG at the "mgmt" Tenant and exported to tenant B. EPG at Tenant B consumes the contract interface. Can't SSH the APICs or switches from a VM in Tenant B. Am I missing something?
Thanks.
11-29-2019 07:41 PM - edited 11-30-2019 06:02 AM
OK. Took me a couple of hours but I've cracked it. It will take as long again to write it up. [Edit: Family Christmas do got in the way. Nearly finished writing up, but it's 1:00am - will finish when I wake. The secret is creating an extra EPG in the mgmt tenant, gving it an IP address and making it Shared Between VRFs. All will be clear when the write up is finished]
11-30-2019 07:47 PM - edited 11-30-2019 10:38 PM
Hi @Antonio Macia ,
OK, I worked through it (making a couple of my own mistakes on the way) and got it working. I documented the complete process on my blog, but carrying on from:
a simple contract that allows SSH only:
This contract is provided by the inband EPG at the "mgmt" Tenant and exported to tenant B. EPG at Tenant B consumes the contract interface. Can't SSH the APICs or switches from a VM in Tenant B.
The bit you were missing is actually not so much what YOU were missing, but what ACI is missing, and that is the ablity to mark the inband EPG subnet as Shared Between VRFs.
The kludge I used to get around this was to create an Application Profile and EPG in the mgmt tenant, added the inband management IP subnet and checked the shared between VRFs option, and had it also provide the contract.
That allowed the route for the inband management subnet to leak into Tenant B, and it all came good.
Don't forget to check out the complete story.
I hope this helps
Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem
Footnote: Regarding the inability to set the Shared between VRFs option on the Inband EPG - I tried forcing the attribute by posting this to the EPG:
{ "fvSubnet": { "attributes": { "annotation": "", "ctrl": "nd", "descr": "", "ip": "192.168.99.1/24", "name": "", "nameAlias": "", "preferred": "no", "scope": "private,shared", "virtual": "no" } } }
into the mgmt tenant - the post worked, and I could see the changed attribute in the object nrowser, but it made no difference to the behaviour. So to fix this properly needs more than just adding the abilty to set the option.
12-01-2019 10:28 AM - edited 12-01-2019 10:42 AM
There is a simpler way to let EPG in another TENANT:VRF to get to inb. The issue is a route leak from inb to the client's VRF that needs an EPG subnet, not just a BD subnet. That is a bit different to config. in MGMT / INB epg compared to the "normal" EPG.
So, without dealing with EPG subnet: create global contract for ICMP in MGMP tenant apply it as provider to INB EPG and export to client's tenant and apply contract interface to clients VRF. Cleate . contract for ICMP in client's tenant, apply as prover to client's EPG and export it to MGMT and apply contact interface to INB EPG. That will take care or route leak both directions.
Also, i it is nothing wrong, i believe, to allow ping between client's IPs and inb IPs. Could be helpful in future.
Now you can add your SSH contact as client's EPG consumer and INB EPG provider.
If you really want to make SSH one way only, in subject un-check "reversed port" and "both direction" and make 2 rules in the subject:
Consumer-provider source TCP (default) 1024-65535 to TCP 22
Provider Consumer TCP (stateful, Established ) TCP/22 to TCP/1025-65535
12-01-2019 11:40 PM
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide