Is two Physical APICs plus one vAPIC a supported configuration?

[RedNectar]

Hi experts,

I had a lab set up as the standard Mini ACI Fabric and Virtual APICs as per the official document.

But because I had a "spare" physical APIC, last time I built the lab I thought I'd try Two physical and one vAPIC.

But the vAPIC just doesn't seem to want to join, and I'm at a loss as how to troubleshoot exactly what is going on. I'm beginning to think that perhaps the system is geared to determining whether the setup is a Standard 3 Physical APIC topology or a Mini ACI Fabric topology is based on whether APIC#2 is physical or virtual.

Most of the troubleshooting documents I've found talk about making sure the Infra VLAN and LLD are configured correctly - LLDP is irrelevant in this situation because the stubborn APIC is a vAPIC

FWIW - this is the situation:

• I am using ACI version 4.2(7f) - the recommended version for topologies with 1generation switches
• APIC#1 and APIC#2 joined the fabric as per normal
• APIC#3 (the vAPIC) was configured on an ESXi host using a standard switch 
• APIC#3 (the vAPIC) has full connectivity on the infra VLAN (can ping 10.0.0.1 and 10.0.0.2) 
• APIC#3 (the vAPIC) has full connectivity on the OOB network (can ping APIC#1 and APIC#2) 
• Both APIC#2 and #3 show two instances of error F1410 
  • One instance lists the affected object as:
    • topology/pod-1/node-1/lon/clSzEqObst-2-expand-no_expansion_contender-3-none-undefined-undefined
  • The other instance lists the affected object as: 
    • topology/pod-1/node-2/lon/clSzEqObst-2-expand-cluster_is_stuck_at_size_2-undefined-none-undefined-undefined
  • Both instances of fault F1410 offer the following description
    • Cluster cannot continue to Expand from its current size 2 for the following reason: No expansion contender. The controller ID is 3, the service ID is None, the internal database partition (shard) ID is Undefined and the replica ID is Undefined
• Additionally, APIC#2 shows four instances of fault F1419, one for each of the following services:
  • proc-nomad, proc-kron, proc-nomad_client and proc-consul - all have the equivalent description of:
    • Service [nomad|kron|nomad_client|proc-consul] failed on apic apic2 of fabric LAB with a hostname apic2

So my initial question is:

• Is two Physical APICs plus one vAPIC a supported configuration?

But implied is also:

• Has anyone successfully setup an ACI fabric with two Physical APICs plus one vAPIC?
• and if so, do you have any suggestions?

Note: Being Lab equipment it is not under any support contract, so callingTAC is not an option.

[ecsnnsls]

Hi,

You have mentioned that you have taken care of LLDP but I am not sure if you already did this. Did you disable LLDP on the VICs of the physical APICs?

[RedNectar]

Hi @ecsnnsls ,

I'm not sure if enabling/disabling LLDP on the VICs of the physical APICs would have anything to do with the situation. Both physical APICs have joined the cluster - quite happily except they can't find APIC3

I really can't find any troubleshooting info about the processes that a vAPIC goes through to join a cluster.

[ecsnnsls]

Yes, that's true. There's very little documentation available. For some reason I was thinking APIC2 didn't join the cluster. My bad!

Couple of things off the checklist,

- Did you check the time synchronization on all three apics and the esxi? 

- Also, if you had deployed the VM using OVA, did it take more than 60 mins for the VM to get deployed. In this case we need to use a new passphrase. 

Sorry, this is all I could scavenge from the document. Perhaps someone with more experience in vAPIC has the solution!

[RedNectar]

Hi @ecsnnsls ,

Good points - and I forgot to mention these in the original:

• Time synchronisation is perfect between the physical APICs and the vAPIC
• I've wasted so much time on this I can update the passphrase with my eyes closed (almost)

Thanks for trying.

[kennethchyee]

Hi @RedNectar ,

may i know any solution for your problem? i am facing the same issue.... thanks.

[RedNectar]

Hi @kennethchyee ,

---

may i know any solution for your problem? i am facing the same issue.... thanks.

---

We gave up and used a physical APIC!

Sorry I couldn't help!

[Robert Burns · Accepted solution]

Dual pAPICs + 1 vAPIC is not supported.  I'm trying to verify if there's any software check preventing this, but regardless not supported as of today.

Robert