Initial Cisco APIC Setup
When the Cisco Application Policy Infrastructure Controller (Cisco APIC) is launched for the first time, the Cisco APIC console presents a series of initial setup options.
Beginning with Cisco APIC release 6.0(2), the initial cluster set up and bootstrapping procedure has been simplified with the addition of GUI screen(s) for cluster bring up.
The GUI supports both the scenarios. A major advantage of using the APIC Cluster Bringup GUI is that, you do not need to enter the parameters for every APIC in a cluster. One APIC can relay the information to the other APICs of the cluster.
- For the physical APIC cluster, configure the Out of Band (OOB) address for APIC 1. Ensure that the CIMC addresses of APICs 2 to N (where N is the cluster size) are reachable via the OOB address of APIC 1.
- Connectivity between out-of-band and the CIMC is mandatory.
- CIMC and APIC subnets could be part of distinct network segments.
Bootstrap procedure
The procedure for new fabric staging and configuration is pretty straight-forward - you set up IPv4 address for APIC1 oobmgmt interface, connect to GUI and follow the wizard, it will ask you for info about fabric topology and config. There are several ways to interact with APIC:
- GUI wizard
- bootstrap API
- cli via CIMC
APIC first boot
Once your server comes up you connect to console or use CIMC IP, use below guide to connect to APIC console
https://unofficialaciguide.com/2018/02/16/using-serial-over-lan-sol-on-the-cimc-to-access-the-apic-instead-of-kvm-console/
You should see the invitation for initial setup - here you can configure oobmgmt IPv4 address to use API/GUI or feed the JSON line with the payload containing cluster and controllers configuration.
Press any key to continue...
Starting Setup Utility
APIC Version: 6.0(2h)
Welcome to APIC Setup Utility
Press Enter Or Input JSON string to bootstrap your APIC node.
If you hit enter - APIC asks you to configure IP address for oobmgmt interface and ADMIN password:
admin user configuration ...
Enter the password for admin [None]:
Reenter the password for admin [None]:
Out-of-band management configuration ...
Enter the IP Address [192.168.10.1/24]: x.x.x.x/25
Enter the IP Address of default gateway [192.168.10.254]: x.x.x.1
Would you like to edit the configuration? (y/n) [n]:
System pre-configured successfully.
Use: https://x.x.x.x to complete the bootstrapping
GUI wizard "Bringing up the Cisco APIC Cluster Using the GUI"
Open a browser and go the APIC GUI
You have to enter the password; if you left the password empty - you have to enter "None" as password.
The wizard is simple - asking you for the topology and values to use.
On the next section you would need to define your APIC controllers, wizard assumes that it runs on APIC1 and assigns ID1 to local appliance. The wizard requires all APIC controllers CIMC password to be entered and validated, once you create all entries you can proceed to Summary page and submit the configuration.
Troubleshooting:
If the validation does not succeed at this stage, we need to verify the following
no SSH access for APIC at this point, and SCP is also unavailable.
[+] Make sure there is reachability between CIMC and APIC
[+] use browser inspector tools to examine the requests generated by APIC.
In the above example, browser inspector tool confirmed we generate the API verify then it timed out “”504 Gateway time-out”
/api/workflows/v1/controller/verify
it means reachability issues, capturing at the GW will be action plan
check MTS, enable jumbo frames at management network, usually CIMC has jumbo MTU enabled at mgmt interface which might cause the below retransimission to happen
1417 2023-11-14 05:23:20.102940181 x.x.x.x ? x.x.x.x TCP 120 [TCP Dup ACK 1406#1] 49328 ? 22 [ACK] Seq=602 Ack=739 Win=64128 Len=0
1418 2023-11-14 05:23:20.102945186 x.x.x.x ? x.x.x.x TCP 124 [TCP Dup ACK 1406#2] 49328 ? 22 [ACK] Seq=602 Ack=739 Win=64128 Len=0
1420 2023-11-14 05:23:20.311576656 x.x.x.x ? x.x.x.x TCP 1570 [TCP Retransmission] 22 ? 49328 [ACK] Seq=739 Ack=602 Win=43080 Len=1456
1421 2023-11-14 05:23:20.746485416 x.x.x.x? x.x.x.x TCP 1570 [TCP Retransmission] 22 ? 49328 [ACK] Seq=739 Ack=602 Win=43080 Len=1456
1429 2023-11-14 05:23:21.586555376 x.x.x.x ? x.x.x.x TCP 1570 [TCP Retransmission] 22 ? 49328 [ACK] Seq=739 Ack=602 Win=43080 Len=1456
1446 2023-11-14 05:23:23.266549466 x.x.x.x? x.x.x.x TCP 1570 [TCP Retransmission] 22 ? 49328 [ACK] Seq=739 Ack=602 Win=43080 Len=1456
1517 2023-11-14 05:23:26.626502870 x.x.x.x ? x.x.x.x TCP 1570 [TCP Retransmission] 22 ? 49328 [ACK] Seq=739 Ack=602 Win=43080 Len=1456
34 1680 2023-11-14 05:23:33.346473067 x.x.x.x ? x.x.x.x TCP 1570 [TCP Retransmission] 22 ? 49328 [ACK] Seq=739 Ack=602 Win=43080 Len=1456
35 1806 2023-11-14 05:23:46.786481719 x.x.x.x ? x.x.x.x TCP 1570 [TCP Retransmission] 22 ? 49328 [ACK] Seq=739 Ack=602 Win=43080 Len=1456
.
The solution involved changing the MTU at the gateway. Reviewing the logs, it was observed that there were TCP retransmissions occurring, potentially due to the mismatch in MTU settings.
To address this issue, the recommendation is to check MTS, enable jumbo frames at the management network, and verify the MTU settings at CIMC's management interface. It's noted that having jumbo MTU enabled at the CIMC management interface could lead to retransmission issues as seen in the logs.
References:
Hi @mabushei ,
Great post. It would be much more readable if you edit it and make your images full width, as explained in this tip:
When posting on the forum, add your pictures inline - i.e. PASTE your picture right where you want it. If it is a screenshot, you'll probably then want to click on the image and make the image large - like this.
This means you pictures are actually SEEN (a) in the email that gets sent to subscribers and (b) anyone who looks at this post in the future. Adding pictures as attachments... puts your submission into the TL;DR category.
