This document describes the steps to be followed to add a new leaf or spine switch to the ACI fabric.
When you unbox the new switch, note down the serial number of the switch. Power on the switch and connect a console to check if it is running in ACI mode or NxOS mode. If running in NxOS mode, follow the steps documented in Converting the switch from NxOS to ACI mode to convert the switch to ACI mode.
Once you confirm the switch is in ACI mode, follow below steps.
1. From the new switch console run the command "setup-clean-config.sh" and reload (Run the
command reload) to cleanup any existing configurations on the switch. This will prevent issues due to
some existing configurations in new switch conflicting with existing fabric, even if the new switch was
configured with another ACI fabric before.
2. Rack the new switch and power it on
3. Connect the switch to the fabric. If you are adding the leaf switch, make sure that the leaf switch is connected to all the spine switches in the fabric. If you are adding a spine switch, make sure to connect the spine switch to all the leaf switches in the fabric.
Considering, the switch is in ACI mode and you have connected it to the fabric, the fabric should now discover the new switch automatically using LLDP.
4. Go to ACI GUI -> Fabric -> Inventory -> Fabric Membership and look for a the new switch which doesn't have any IP address assigned (0.0.0.0) and no node ID assigned. Please confirm the new switch by verifying the serial number.
5. Right click on the new switch and click "Register Switch". Now you will see few editable fields. It is very important to fill right information for below fields. Rest of the fields can be left to default.
- POD ID: Default is 1. You need to change this to right POD ID if you have a multi-pod fabric.
- Node ID: It is very important to configure the right node ID. Once you assign and it gets registered, you cannot change this without decommissioning the switch.
- Node Name: Enter the name for the node.
In this example, I am registering the new switch with node ID 103 and node name leaf103.
6. Click "Update" and wait for the APIC to assign a TEP IP to the new switch.
7. You can verify the switch status in GUI -> Fabric -> Inventory -> Topology. You can see new switch part of topology now.
8. SSH to the APIC and run the command "acidiag fnvread" to confirm the new switch shows up as "active"
Scenario 2: The newly added switch shows as "not supported"
In ACI GUI -> Fabric -> Inventory -> Fabric Membership page if the new switch is listed as "no" under "Supported Model" column, this could be the issue of your APIC catalog firmware is too old and doesn't have the model of new switch listed in there. To solve this, upgrade the APIC to the same version level as the new switch. After that the new switch should be able to join the fabric.
Scenario 3: SSL certificate issue
- If the switch fails to get registered with the fabric after you assign a node ID and node name, there could be SSL certificate issue. You can verify the same using below method.
- From the console, run the command "netstat -an | grep <TEP ip of APIC>" and check for a "ESTABLISHED" session with on port 12215 with APIC. This session could be established with any of the APIC in your fabric. So re-run the command with different APIC IP's
- Below is an example of the above step.
-
Established session with any of the APIC on port 12215 means the new switch is able to communicate with the APIC policy manager. If you don't see this session with any of the APIC, it could be a SSL certificate issue. Open a case with TAC for further assistance.
