ACI Multipod Fabric Discovery

[Ziad El Achkouty]

Hello,

I have configured 2 IPNs and POD2 on ACI with tep pool 10.1.0.0/16

OSPF between spine and IPN is up

I can see the pod2 spine in the discovered fabric however, its status is stuck at discovering and no infa IP is assigned to it.

[shehreyarkhan]

Hi, 

How did you solve the problem, I am facing the same issue.

Regards

[e-chuah]

Hi,
We managed to solve it. The issue we encountered was OSPF MD5 authentication.
we tested in the lab, below are the findings
when you use OSPF MD5 authentication in multipod scenario:
(1) During initial remote pod spine discovery, you can configure MD5 in APIC l3out.
(2) But do not configure MD5 authentication on the interface between the remote spine and the IPN,
(if u do, your IPN-remoteSpine OSPF neighbor will be stuck in INIT state.)
(3) The remote pod spine will be discovered (with no MD5 authentication on the IPN-remoteSpine interface)
OSPF neighbor between IPN-remoteSpine interface will come up.
(4) Put back the OSPF MD5 authentication in the IPN-remoteSpine interface.
(5) OSPF neighbor can then come up with MD5 Authentication in place.
There is no need to manually install public key in the APIC.
THe above is tested using APIC 4.2.6d was due to OSPF MD5 authentication.
We did not use the Multipod wizard to do the configuration. This is because the wizard does not allow us to change the spine interface profile naming convention.
If you use the wizard default settings and naming convention, it is very confusing and it created a lot of unnecessary policy which makes things look complicated.
We configure the Multipod manually without using the Wizard.

[shehreyarkhan]

Hi,

I am not using any authentication and no public keys on APIC. I have also erased the config of the remote spine and leaf but still the same issue.

[e-chuah]

ACI Troubleshooting: Multipod - DGTL-BRKACI-2934

https://www.ciscolive.com/global/on-demand-library.html?search.event=ciscoliveus2020&search=aci%20multipod#/session/1573153536777001JSmV
by Joseph Young
is very good. You may want to look at this to see if helps.
So your scenario is remote spine remain in "Nodes Pending Resigration" tab and stucked in "discovering"?
Or the remote spine went to the "Registered Nodes" tab but stucked in inactive state?
A few things you may want to take note:

* Remote spine can only be discovered if you have leaf connected to that spine.
* If there is cabling error between your IPN and remote spine, you will also have issue.
* From APIC, make sure you can ping the IPN ip (the one facing the remote spine).
* bash
* ping -I bond0.
* this will ensure connectivity between APIC and IPN for DHCP discover and offer to happen.

[Hector Gustavo Serrano Gutierrez]

Hi @e-chuah,

One reason that the APIC is not replying with the DHCP Offer is that the Spines' uplinks of the 2nd Pod are not already configured as part of the infra L3Out. This L3Out is the one you'll find in Tenant Infra and you should have in advance the Spines of your 2nd Pod (and subsequent Pods in the Multi-Pod architecture) there under Configured Nodes and the respective interfaces facing the IPN configuration under Logical Interface Profiles, etc.

Also, make sure you have all related Fabric Access Policies in place and of course the 2nd Pod's TEP pool in Fabric > Inventory > Pod Fabric Setup Policy.

One good source of hits to troubleshoot ACI is Troubleshooting Cisco Application Centric Infrastructure, 2nd Edition

Hope this helps.

[IsaacGV]

Hi Ziad

Was the problem solved?

If not, you can set up again Multi Pod, erase all configs and TEP Pool for Pod 2.

Erase all configuration from your Spine 201 and Leaf X with setup-clean-config. Before check if the version of your devices are the same.

Check IPs for your IPN connection, and check if your IPN devices support: 

• Multicast
• PIM BIDIR
• DHCP Relay
• OSPF
• Increased MTU
• Layer 3 sub-interfaces

When you set up IPN make sure that you configure Rendezvous Point Address on IP PIM command and the IP for Multicast that you configure should be the same that you configured on the APIC initialization.

The IPN interfaces that connect directly with SPINE should be sub-interfaces with VLAN 4 encapsulation, also verify MTU.

You need two pools when you are configuring Multi Pod, 1 pool for EXTERNAL TEP on Pod 1 (this pool should be between /22 - /27 prefix mask) an other pool for External TEP for Pod 2.

You should use the set up wizard for configure Multi Pod is a great guide. 

https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-739714.html