ACI APIC setup

[ksherwood]

Hi all,

quick question:

When you configure the APICs one by one, and once you have configured two of them, are you supposed to see the second one in the GUI topology view once you have browsed and logged into the first ?

Note: There is no Spine or Leaf connected yet as the APICs are only connected together on the same management network.

Thanks in advance.

[Tomas de Leon]

What N9K model are you having issues configuring OOB mgmt address on Spine?

you do not need to manually configure these on the switch itself. Policy should be passed to the N9K.

From the APIC, configure 1.2.3.4" for the SPINE OOB and then get the logs from the following:

zgrep -E "1.2.3.4" *

where "1.2.3.4" is the IP address trying to apply to Spine

Switch:
/var/log/dme/log

APIC:
/var/log/dme/log

Also get from switch:

 "ifconfig -a eth0"

 "ip route show | grep eth0"

Note: you can access the switch via "attach" on the APIC itself. SSH to APIC and then "attach spine1".  You do not have to access via console.

[ksherwood]

Thanks Tomas,

The Spine is an N9K-C9336PQ

I have already tried to configure the Spine from the out of band management process via the GUI. I will now get the logs.

I have already checked the interface and routes after GUI configuration. The interface and route table does not include my GUI configuration. Once I use the CLI via console they do.

Thanks for the "attach" information.

All I really need is the CLI process of configuring the mgmt IP address. The GUI obviously has issues.

[Tomas de Leon]

There is something that we are missing.  If the OOB mgmt addresses cannot be configured via the APIC there should be a "fault" listed.

From the APIC:

" show faults spine  <nodeId> "

Also, if you have console access and a maintenance window, you can run "setup-clean-config.sh" on one spine and then reload.  This will erase the spine confguration and then let the APIC reconfigure it.  The will ensure there is no "stale" configuration that may be in conflict.

T.

[ksherwood]

Ok, thanks guys. I'll run the commands.

I will clean and restart. It's all in test phase anyway.

One thing I did wonder is that when the APIC first sees the leaf and spine, do you have to right click "register" or is the device automatically registered because the APIC has listed it ???

Do the Leaves and Spines really need to be exactly the same IOS ? We bought them all together.

[Tomas de Leon]

Yes, the all the APICs and Switch nodes need to be running the same ACI version.  Objects may be added or removed between version.  Features are added in each version so they need to match.  Also, there is a configuration setting that may be set that only allows nodes with a certain version to join the fabric.  

When the leaf and spine nodes go thru the discovery process, the APIC will see them in the fabric membership inventory.  This is where you assign a NODEID and NAME for the switch.  After that, the APIC assigns the node an IP address from the INFRA TEP Address pool.  Then, the leaf or spine is registered.  There is no right clicking to register.

From the APIC, please provide the display output of the commands "acidiag fnvread" and "show version"

Also, the links and videos above are goo resources to understand the fabric bring up process.

Thanks

T.

[ksherwood]

Ok, so the Spines did register and was given ID and hence TEP addressing. Would this still happen if the IOS wasn't compatible ?

[Tomas de Leon]

You can set the specific firmware for node switches and if the firmware was not equal to the set version they would not be able to join.  Most fabric admin to not configure this parameter or configure "All" so all versions can join.  That said, All node switches need to run the matching version to the APIC. If they are not, you are operating the fabric in an "UNSUPPORTED" configuration.  Also, the fabric may not behave as expected because of the difference in INXOS..

[ksherwood]

ok, thanks Tomas, I'll check.

[ksherwood]

Ok, so I checked the firmware and it looks the same.

Controller = 1.2(1i)
Spine = N9000-11.2(2g)
Leaf = N9000-11.2(2g)

BTW the attach command from the Apic doesn't work !

I think I will clean the Spine next and re-register

"setup-clean-config.sh"

ok, so I cleaned Spine 1 and reloaded it and it appears to be fixed without even re-registering from the APIC. Spine 2 needed a clean and appears to be working now as well i the same manner.

[Tomas de Leon]

you do not have to re-register with a clean wipe on the switches.  The switch configuration is managed by the APIC.  The clean switch reboots and downloads the config from the APIC.

But you still have an issue with a mismatch of versions between APIC and the Switches.  You are running "i" on the APIC and "g" on the switches.  I would suggest upgrading the fabric to 1.2(3h) or 1.3(2f).

But I am glad things are working now...

T.

[ksherwood]

How do you match the firmware from APIC to fabic ?

ie APIC = 1.2(1i) and fabric = 11.2(2g)

[dpita]

Hello

Whenever you are looking at controller firmware, there will be a section for "related software" where you will see the switch image. 

The easiest way is just to match the letters at the end. 

1.2(1i) matches 11.2(1i) there is a pattern. APICs have the format #.#(#x) and switches have ##.#(#x)

[ksherwood]

Thanks Daniel, that's what I was looking for, so I need to match both number and letter to be exact.

[dpita]

Hello

To further what Tomas is saying. It shouldn't be this complicated there must be something we are missing causing this issue. 

If you get a chance could you provide the following output from the spine console:

cat /mit/sys/summary

I want to make sure the spine was discovered, has a name and is active. 

Also, is everything in the fabric on the same version?

[ksherwood]

Thanks in advance Robert