|
Device Replacement in SD-Access
Contents
Author
Cheeho Yan
Senior Technical Leader
Executive Summary
During network operation, network administrators may need to replace network devices that become faulty with identical hardware. Being able to perform such tasks, smoothly, and with minimum network downtime is critical to meeting the requirement of network service continuity.
This document outlines steps to replace device playing various SD-Access fabric roles using the Cisco DNA Center RMA workflows feature. These procedures provide examples and procedures to help network architects, operation engineers, and site support technicians address faulty devices in production networks.
To prepare for the document, a lab was set up to perform like-for-like device replacement for the following roles in a typical SDA deployment:
- Fabric Access Points
- Fabric Border/Control Plane Nodes
- Fabric Extended Nodes
- Fabric Edges and their components
- Fusion (Outside the Fabric)
- Switch Stack member
Cisco DNA Center Inventory Overview
The Inventory function retrieves and saves details, such as host IP addresses, MAC addresses, and network attachment points about devices in its database.
The Inventory feature can also work with the Device Controllability feature to configure the required network settings on devices, if these settings are not already present on the device.
After the initial discovery, Cisco DNA Center maintains the inventory by polling the devices at regular intervals. The default interval is every 24 hours. However, you can change this interval as required for your network environment. Also, a configuration change in the device triggers an SNMP trap, which in turn triggers device resynchronization. Polling occurs for each device, link, host, and interface. Only the devices that have been active for less than one day are displayed. This prevents stale device data, if any, from being displayed. On average, polling 500 devices takes approximately 20 minutes.
Cisco DNA Center RMA Workflow
Feature Summary
The Return Material Authorization (RMA) workflow enable network administrators to replace failed devices quickly. RMA provides a common workflow to replace routers, switches, and APs.
When using the RMA workflow with routers and switches, the software image, configuration, and license are restored from the failed device to the replacement device. For wireless APs, the replacement device is assigned to the same site, provisioned with primary wireless controller, RF profile, and AP group settings, and placed on the same floor map location in Cisco DNA Center as the failed AP. For Cisco Switch stacks (hardware stacking), a network administrator does not need to follow a separate procedure in Cisco DNA Center for member switch replacement, it is handled by the active switch. The member switch is replaced by the active switch by providing software image and configuration. Full stack replacement is handled by Cisco DNA Center.
Prerequisite
In order to perform RMA workflow, the following requirement need to be met:
- The software image version of the faulty device must be imported in the image repository before marking the device for replacement.
- The faulty device must be in an unreachable state.
- If the replacement device onboards Cisco DNA Center through Plug and Play (PnP), the faulty device must be assigned to a user-defined site.
- The replacement device must not be in a provisioning state while triggering the RMA workflow.
- For switch stacks replacement, the number of stacks for faulty and replacement device should be same.
One-Touch and Zero-Touch RMA Workflow
- One-touch onboarding of replacement device is supported starting from Cisco DNA Center version 1.3.1.
https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/1-3-1/release_notes/b_cisco_dna_center_rn_1_3_1.html
- Zero-touch onboarding of replacement device through PnP is supported for fabric and LAN automation devices staring from Cisco DNA Center 2.3.3.
https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-3-3/release_notes/b_cisco_dna_center_rn_2_3_3.html?dtid=osscdc000283
Zero-touch RMA works in SD-Access Fabric in the following way:
- RMA service creates a temporary DHCP server in neighbor device while marking the device for replacement if device is part of Fabric.
- When a user connects the replacement device to network, it can get the DHCP IP from this temporary DHCP server.
- Using this IP address obtains from the DHCP server, the device can be onboarded to Cisco DNA Center via PnP.
- The temporary DHCP server is deleted from neighbor device once the device is replaced.
- After marking the device for replacement, in an event that the user unmarks the device, temporary DHCP server will be deleted from neighbor device.
- When an Extended node is to be replated, there is no need to create the DHCP server, as the Extended node IP address is given by the user’s DHCP server.
Limitations of RMA Workflow
The following limitations apply to DNA Center 2.3.5:
- RMA supports replacement of similar devices only. For example, a Cisco Catalyst 3650 switch can be replaced only with another Cisco Catalyst 3650 switch. Also, the platform IDs of the faulty and replacement devices must be the same.
- RMA supports replacement of all switches, routers, and Cisco SD-Access devices, except for the following:
- Devices with embedded wireless controllers
- Cisco Wireless Controllers
- Chassis-based Nexus 7700 Series Switches
- Switch stacks (SVL stacking)
- RMA supports devices with an external SCEP broker PKI certificate. The PKI certificate is created and authenticated for the replacement device during the RMA workflow. The PKI certificate of the replaced faulty device must be manually deleted from the certificate server.
- The RMA workflow supports device replacement only if:
- Both the faulty and replacement devices have the same extension cards.
- The number of ports in both devices does not vary because of the extension cards.
- The faulty device is managed by Cisco DNA Center with a static IP. (RMA is not supported for devices that are managed by Cisco DNA Center with a DHCP IP, except extended node and AP in fabric.)
- Fabric edge replacement does not support the DHCP server configuration in the neighbor device if the neighbor device is not part of the fabric. Because intermediate nodes are not part of the Cisco SD-Access fabric, the DHCP server with option 43 is not pushed.
- Make sure that the replacement device is connected to the same port to which the faulty device was connected.
- Cisco DNA Center does not support legacy license deployment.
The RMA workflow deregisters the faulty device from Cisco SSM and registers the replacement device with Cisco SSM.
- If the software image installed on the faulty device is earlier than Cisco IOS XE 16.8, the License Details window does not display the Network and Feature License details and no warning message is displayed. Therefore, you should be aware of the legacy network license configured on the faulty device and manually apply the same legacy network license on the replacement device.
- If the software image installed on the faulty device is Cisco IOS XE 16.8 or later, the License Details window displays details of the network license (for example, Legacy or Network) and the feature license (for example, IP Base, IP Service, or LAN Base). The following warning message is displayed while marking the faulty device for replacement:
Some of the faulty devices don't have a Cisco DNA license. Please ensure your replacement device has the same Legacy license of the faulty device enabled.
- If the legacy network licenses of the replacement and faulty devices do not match, the following error message is displayed during the license deployment:
Cisco DNA Center doesn't support legacy license deployment. So manually update the faulty device license on the replacement device and resync before proceeding.
- Cisco DNA Center supports PnP onboarding of the replacement device in a fabric network, except when:
- The faulty device is connected to an uplink device using multiple interfaces.
- LAN automation uses overlapping pools.
- If the replacement device onboards through the PnP-DHCP functionality, make sure that the device gets the same IP address after every reload and the lease timeout of DHCP is longer than two hours.
Lab Overview
Test Summary
Like for like hardware were used to perform device replacement. Hardware tested in this document include:
- Access Points
- 3700 Industrial AP
- 9120 Wifi 6 AP
- Fabric Extended Node
- Fabric Edge Node
- C9300
- C9400
- Supervisor
- Line Card
- Chassis
- Fully Loaded chassis
- Fusion Router
- Border/Control Plane Node
Switch stack member replacement:
Test Methodology
The following typical procedures were carried out:
- Prepare the replacement devices. The replacement device hardware is identical to the devices to be replaced
- Ensure that the devices to be replaced are unreachable by Cisco DNA Center (Except for switch stack member replacement and switch line card replacement)
- Execute RMA workflow.
Software Versions
The document was written using the following products and versions:
|
Product Family
|
Version
|
|
DNAC DN2-HW-APL DNA Center
|
2.3.5.0-70586
|
|
C3560CX Extended Node
|
15.2(7)E6
|
|
C9300 Edge
|
17.9.1
|
|
3700 AP
|
8.10.162
|
|
9120 AP
|
17.9.1
|
|
C9300 Fusion
|
17.9.1
|
|
C9300 FIAB Switch Stack
|
17.9.1
|
|
C9400 Edge
|
17.9.1
|
The tests were performed using Cisco DNA Center version 2.3.5.0-70586. This is a pre-CCO version. It was used because at the time of writing, a CCO version was not yet available. There might be slight changes in GUI, and additional bug fixes. The overall steps were expected to remain unchanged.
- Verify that the replacement devices are in the management state in Cisco DNA Center Inventory.
- Switch Stack member
Extended Node Replacement
Catalyst 3560CX
Topology
Procedures
Step 1. In the main menu, go to Provision > Inventory.
Step 2. Select the device to be replaced. Click Actions Device Replacement > Mark for Replacement.
Step 3. Click Mark to acknowledge the operation.
Step 4. Remove the device to be replaced. Connect the replacement device using the same connections as with the old device.
Step 5. Initiate the Plug and Play process. Below is the reference of the commands entered into the C3560CX:
SFO-C9300-E1#delete flash:vlan.dat
Delete filename [vlan.dat]?
Delete flash:/vlan.dat? [confirm]
SFO-C9300-E1#delete flash:config.text
Delete filename [config.text]?
Delete flash:/config.text? [confirm]
%Error deleting flash:/config.text (No such file or directory)
SFO-C9300-E1#delete flash:private-config.text
Delete filename [private-config.text]?
Delete flash:/private-config.text? [confirm]
%Error deleting flash:/private-config.text (No such file or directory)
SFO-C9300-E1#delete nvram:*.cer
Delete filename [*.cer]?
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Delete nvram:CiscoLicensi#1CA.cer? [confirm]
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Delete nvram:IOS-Self-Sig#1.cer? [confirm]
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Delete nvram:CiscoDNACent#3E88CA.cer? [confirm]
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Delete nvram:sdn-network-#871A.cer? [confirm]
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Delete nvram:sdn-network-#BDCCA.cer? [confirm]
SFO-C9300-E1#config t
Enter configuration commands, one per line. End with CNTL/Z.
SFO-C9300-E1(config)#crypto key zeroize
% All keys will be removed.
% All router certs issued using these keys will also be removed.
Do you really want to remove these keys? [yes/no]: yes
SFO-C9300-E1(config)#
230933: Aug 25 20:30:01.191: %CRYPTO_ENGINE-5-KEY_DELETED: A key named TP-self-signed-2216098374 has been removed from key storage
230934: Aug 25 20:30:01.192: %CRYPTO_ENGINE-5-KEY_DELETED: A key named TP-self-signed-2216098374.server has been removed from key storage
230935: Aug 25 20:30:01.192: %CRYPTO_ENGINE-5-KEY_DELETED: A key named dnac-sda has been removed from key storage
230936: Aug 25 20:30:01.192: %CRYPTO_ENGINE-5-KEY_DELETED: A key named sdn-network-infra-iwan has been removed from key storage
230937: Aug 25 20:30:01.193: %SSH-5-DISABLED: SSH 2.0 has been disabled
SFO-C9300-E1(config)#no crypto pki certificate pool
This will remove all downloaded certificates for the trustpool
Are you sure you want to do this? [yes/no]: yes
SFO-C9300-E1(config)#vtp mode off
Setting device to VTP Off mode for VLANS.
SFO-C9300-E1(config)#vtp mode transparent
Setting device to VTP Transparent mode for VLANS.
SFO-C9300-E1(config)#exit
SFO-C9300-E1#write erase
************************************************************************************************************
Erasing Nvram will not clear license trust code.
************************************************************************************************************
Erasing the nvram filesystem will remove all configuration files! Continue? [confirm]
[OK]
Erase of nvram: complete
SFO-C9300-E1#
230939: Aug 25 20:30:55.446: %SYS-7-NV_BLOCK_INIT: Initialized the geometry of nvram
SFO-C9300-E1#reload
System configuration has been modified. Save? [yes/no]: no
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]
230940: Aug 25 20:31:09.989: %SYS-5-RELOAD: Reload requested by sdaadmin on console. Reload Reason: Reload Command.
Chassis 1 reloading, reason - Reload command
Aug 25 20:31:14.956: %PMAN-5-EXITACTION: F0/0: pvp: Process manager is exiting: reload fp action requested
Aug 25 20:31:16.457: %PMAN-5-EXITACTION: R0/0: pvp: Proceger is exiting: process exit with reload stack code
Step 6. Change the Focus to Device Replacement. Wait till the device to be replaced is in the Ready for Replacement state. Go to Actions > Device Replacement > Replace Device.
Step 7. Under the Unclaimed tab, the replacement should show up ready to be claimed. Select the device and click Next.
Step 8. Select Now. Click Next to start the replacement.
Step 9. After reviewing the serial numbers of the devices, click Replace.
Step 10. Click Monitor Replacement Status to view the progress.
Step 11. In the Device Replacement focus screen, the device should show the In-Progress status. Click on the In-Progress link to view additional details.
Step 12. The Replace Status tab should show each task turning green as it completes successfully. Click Refresh as needed.
Step 13. The replacement now shows up in the inventory with the new serial number.
Access Point Replacement
9120 Access Point
Topology
Procedures
Step 1. Select the device to be replaced. Click Actions > Device Replacement > Mark for Replacement.
Step 2. Click Mark to acknowledge the operation.
Step 3. Remove the AP to be replaced. Connect the replacement AP using the same connection as with the old AP. Set to AP to factory default if it has not been done so already.
Step 4. If needed, on the Edge switch that is connected to the AP, bounce the port that is connected to the AP and ensure that CDP neighbor is showing the new AP.
DFW-C9300-E1(config-if)#int t1/0/3
DFW-C9300-E1(config-if)#no shut
DFW-C9300-E1(config-if)#
001659: Nov 22 20:12:55.272: %ILPOWER-5-DETECT: Interface Te1/0/3: Power Device detected: IEEE PD
001660: Nov 22 20:12:55.544: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/3, changed state to down
001661: Nov 22 20:12:56.256: %ILPOWER-5-POWER_GRANTED: Interface Te1/0/3: Power granted
001662: Nov 22 20:13:18.063: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/0/3, changed state to up
001663: Nov 22 20:13:19.063: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/0/3, changed state to up
DFW-C9300-E1#sh cdp nei
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
D - Remote, C - CVTA, M - Two-port Mac Relay
Device ID Local Intrfce Holdtme Capability Platform Port ID
DFW-C9300-FIAB.sdn-lab.cisco.com
Ten 1/1/3 168 R S I C9300-24U Ten 1/1/8
DFW-C9300-FIAB.sdn-lab.cisco.com
Ten 1/1/4 138 R S I C9300-24U Ten 2/1/8
DFW-C9300-E2.sdn-lab.cisco.com
Ten 1/1/1 136 S I C9300-24U Ten 1/1/1
DFW-C9300-E2.sdn-lab.cisco.com
Ten 1/1/2 178 S I C9300-24U Ten 1/1/2
APF4BD.9EBD.91B8 Ten 1/0/3 118 R T C9120AXI- Gig 0
Rack9_OOB_Mgmt-Switch.sdn-lab.cisco.com
Gig 0/0 164 S I C9200L-48 Gig 1/0/18
Total cdp entries displayed : 6
DFW-C9300-E1#
Step 5. Wait till the AP to be replaced is in the Ready for Replacement state.
Step 6. In the Device Replacement focus, wait till the device to be replaced is in the Ready for Replacement state. Go to Actions > Device Replacement > Replace Device.
Step 7. Under the Unclaimed tab, the replacement AP should be listed ready to be claimed. Select this device and click Next.
Step 8. Select Now. Click Next to start the replacement.
Step 9. After reviewing the serial numbers of the devices, click Replace.
Step 10. Click Monitor Replacement Status to view the progress.
Step 11. In the Device Replacement focus screen, the device should show the In-Progress status. Click on the In-Progress link to view additional details.
Step 12. The Replace Status tab should show each task turning green as it completes successfully. Click Refresh as needed. Close the pop-up window when all the tasks have been completed.
Step 13. The replacement is now showing up in the inventory with the new serial number.
3700 Access Point
Topology
Procedures
Step 1. Select the device to be replaced. Click Actions > Device Replacement > Mark for Replacement.
Step 2. Click Mark to acknowledge the operation.
Step 3. Remove the AP to be replaced. Connect the replacement AP using the same connection as with the old AP.
Step 4. If needed, on the Edge switch that is connected to the AP, bounce the port that is connected to the AP. Onboard the AP.
SFO-C9300-E2(config)#int g1/0/1
SFO-C9300-E2(config-if)#no shut
SFO-C9300-E2(config-if)#
067251: Nov 22 23:53:06.449: %ILPOWER-5-DETECT: Interface Gi1/0/1: Power Device detected: IEEE PD
067252: Nov 22 23:53:07.265: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/1, changed state to down
067253: Nov 22 23:53:08.423: %ILPOWER-5-POWER_GRANTED: Interface Gi1/0/1: Power granted
067254: Nov 22 23:53:11.589: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/1, changed state to up
AP88f0.3160.a980#capwap ap primary-base wlc 172.20.34.100
AP88f0.3160.a980#
*Nov 23 02:08:46.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.20.34.100 peer_port: 5246
*Nov 23 02:08:46.207: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: 172.20.34.100 peer_port: 5246
*Nov 23 02:08:46.211: %CAPWAP-5-SENDJOIN: sending Join Request to 172.20.34.100
*Nov 23 02:08:46.899: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to the reason code 56
*Nov 23 02:08:46.907: %CAPWAP-5-JOINEDCONTROLLER: AP has joined controller SFO-WLC-A
*Nov 23 02:08:46.911: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 172.20.34.100:5246
*Nov 23 02:08:46.915: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to down
*Nov 23 02:08:46.919: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to the reason code 27
*Nov 23 02:08:46.919: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to the reason code 27
*Nov 23 02:08:46.987: %CAPWAP-5-AP_EASYADMIN_INFO: AP Easy Admin information - EASY_ADMIN is not set, turn off easy admin service!
*Nov 23 02:08:47.619: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to the reason code 10
*Nov 23 02:08:47.643: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to up
*Nov 23 02:08:47.915: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down
*Nov 23 02:08:47.951: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to down
*Nov 23 02:08:47.959: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset
*Nov 23 02:08:48.943: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
*Nov 23 02:08:48.951: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to down
*Nov 23 02:08:48.987: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to up
*Nov 23 02:08:48.995: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to down
*Nov 23 02:08:49.003: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
*Nov 23 02:08:49.987: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to up
*Nov 23 02:08:49.995: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down
*Nov 23 02:08:50.023: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Nov 23 02:08:51.023: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
Writing out the event log to flash:/event.log ...
*Nov 23 02:08:58.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.20.34.100 peer_port: 5246
*Nov 23 02:08:58.211: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: 172.20.34.100 peer_port: 5246
*Nov 23 02:08:58.211: %CAPWAP-5-SENDJOIN: sending Join Request to 172.20.34.100
*Nov 23 02:08:58.259: %LWAPP-3-CLIENTERRORLOG: Operator changed mode for 802.11g. Rebooting.
*Nov 23 02:08:58.471: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to the reason code 11
*Nov 23 02:08:58.471: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to the reason code 11
*Nov 23 02:08:58.479: %SYS-5-RELOAD: Reload requested by CAPWAP CLIENT. Reload Reason: Operator changed mode for 802.11g.
Step 5. Wait till the AP to be replaced is in the Ready for Replacement state.
Step 6. Verify that the replacement AP is there in the inventory. This AP should have been onboarded and the information is reported by the WLC. Note that this AP is showing its serial number as device name.
Step 7. In Device Replacement focus. Wait till the device to be replaced is in the Ready for Replacement state. Go to Actions > Device Replacement > Replace Device.
Step 8. Under the Managed tab, the replacement AP should show up. Select the device and click Next.
Note: 3700 AP does not have Plug and Play capabilities. The AP onboarding is performed and controlled by the WLC.
Step 9. Select Now. Click Next to start the replacement.
Step 10. After reviewing the serial numbers of the devices, click Replace.
Step 11. Click Monitor Replacement Status to view the progress.
Step 12. After all the tasks have been completed, the replacement AP is now showing up in the inventory with the new serial number.
Fabric Edge Replacement
Catalyst 9300
Topology
Procedures
Step 1. In the main menu, go to Provision > Inventory.
Step 2. Ensure that the device is in Unreachable state in the inventory. Select the device to be replaced. Click Actions > Device Replacement > Mark for Replacement.
Note: There would be use cases that the device to be replaced is reachable through IP. It is suggested to shut down this device or disconnect the connection to the device and then resync in Cisco DNA Center to make this device in Unreachable state. This would ensure that the RMA workflow will proceed successfully.
Step 3. Click Mark to acknowledge the operation.
Step 4. In Device Replacement focus, the device will show as Network Readiness Requested.
Step 5. Review the upstream devices’ interface configurations towards the device to be replaced. These interface configurations will not be changed.
SFO-C9300-BN1#sh run int g1/0/37
Building configuration...
Current configuration : 308 bytes
!
interface GigabitEthernet1/0/37
description Fabric Physical Link
no switchport
dampening
ip address 172.20.10.2 255.255.255.254
no ip redirects
ip pim sparse-mode
ip router isis
load-interval 30
shutdown
bfd interval 250 min_rx 250 multiplier 3
clns mtu 1400
isis network point-to-point
end
SFO-C9300-BN2#sh run int g1/0/37
Building configuration...
Current configuration : 298 bytes
!
interface GigabitEthernet1/0/37
description Fabric Physical Link
no switchport
dampening
ip address 172.20.10.9 255.255.255.254
no ip redirects
ip pim sparse-mode
ip router isis
load-interval 30
bfd interval 250 min_rx 250 multiplier 3
clns mtu 1400
isis network point-to-point
end
Step 6. Wait till the device to be replaced is in the Ready for Replacement state.
Step 7. Go to Actions > Device Replacement > Replace Device.
Step 8. It can be observed that a DHCP server is configured by Cisco DNA Center in the upstream switch.
SFO-C9300-BN2#sh run | i dhcp
ip dhcp excluded-address 172.20.10.1 172.20.10.7
ip dhcp excluded-address 172.20.10.9 172.20.10.255
ip dhcp pool GigabitEthernet1/0/37
ip dhcp class ciscopnp
class-map match-any system-cpp-police-dhcp-snooping
ip address dhcp
snmp-server enable traps dhcp
SFO-C9300-BN2#
Dec 10 01:49:50.239: %SYS-6-LOGOUT: User sdaadmin has exited tty session 1(172.16.0.71)
SFO-C9300-BN2#sh run | s dhcp
ip dhcp excluded-address 172.20.10.1 172.20.10.7
ip dhcp excluded-address 172.20.10.9 172.20.10.255
ip dhcp pool GigabitEthernet1/0/37
network 172.20.10.0 255.255.255.0
option 43 ascii 5A1D;B2;K4;I172.16.0.70;J80;
default-router 172.20.10.9
class ciscopnp
address range 172.20.10.1 172.20.10.254
ip dhcp class ciscopnp
option 60 ^ciscopnp
class-map match-any system-cpp-police-dhcp-snooping
description DHCP snooping
ip address dhcp
snmp-server enable traps dhcp
SFO-C9300-BN2#
Step 9. Type in “pnp service reset” in the replacement device to trigger PnP for a 9300 that is on and after IOS-XE 16.12. For a release prior to 16.12, refer to https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/tech_notes/b_dnac_sda_lan_automation_deployment.html for the commands to use.
SFO-C9300-E2#pnp service reset
PnP Reset command will erase config and reload the device.
Are you sure you want to continue? [yes/no]: y
090229: .Dec 10 01:46:26.432: %SYS-5-CONFIG_P: Configured programmatically by process Exec from console as vty0
090230: .Dec 10 01:46:26.458: %SYS-5-CONFIG_P: Configu
FIPS: Flash Key Check : Key Not Found, FIPS Mode Not Enabled
cisco C9300-48U (X86) processor with 1309516K/6147K bytes of memory.
Processor board ID FOC2214Q0S3
2048K bytes of non-volatile configuration memory.
8388608K bytes of physical memory.
1638400K bytes of Crash Files at crashinfo:.
11264000K bytes of Flash at flash:.
Base Ethernet MAC Address : 50:61:bf:ec:0f:00
Motherboard Assembly Number : 73-17957-07
Motherboard Serial Number : FOC22124M9D
Model Revision Number : E0
Motherboard Revision Number : B0
Model Number : C9300-48U
System Serial Number : FOC2214Q0S3
CLEI Code Number :
No startup-config, starting autoinstall/pnp/ztp...
Autoinstall will terminate if any input is detected on console
Autoinstall trying DHCPv4 on GigabitEthernet0/0
Autoinstall trying DHCPv6 on GigabitEthernet0/0
--- System Configuration Dialog ---
Would you like to enter the initial configuration dialog? [yes/no]: Dec 10 01:50:18.084: %IOX-3-IOX_RESTARTABITLITY: R0/0: run_ioxn_caf: Stack is in N+1 mode, disabling sync for IOx restartability
Autoinstall trying DHCPv6 on GigabitEthernet0/0,Vlan1
Autoinstall trying DHCPv4 on GigabitEthernet0/0,Vlan1
Acquired IPv4 address 172.20.10.8 on Interface Vlan1
Received following DHCPv4 options:
vendor : 5A1D;B2;K4;I172.16.0.70;J80;
OK to enter CLI now...
pnp-discovery can be monitored without entering enable mode
Entering enable mode will stop pnp-discovery
Guestshell destroyed successfully
PnP Discovery trying to connect to PnP server (http://172.16.0.70:80/pnp/HELLO)
PnP Discovery connected to PnP server (http://172.16.0.70:80/pnp/HELLO)
Press RETURN to get started!
*Dec 10 01:52:00.306: %PKI-6-TRUSTPOINT_CREATE: Trustpoint: TP-self-signed-2453152256 created succesfully
*Dec 10 01:52:00.716: %PNP-6-PNP_RELOAD_INFO_STOPPED: Reload reason (PnP Service Info 2413-PnP reset CLI) stopped by (profile=pnp-zero-touch, ip=172.16.0.70, port=80)
*Dec 10 01:52:00.716: %PNP-6-HTTP_CONNECTED: PnP Discovery connected to PnP server (profile=pnp-zero-touch, ip=172.16.0.70, port=80)
*Dec 10 01:52:00.898: %CRYPTO_ENGINE-5-KEY_ADDITION: A key named TP-self-signed-2453152256 has been generated or imported by crypto-engine
*Dec 10 01:52:00.955: %PKI-4-NOCONFIGAUTOSAVE: Configuration was modified. Issue "write memory" to save new IOS PKI configuration
*Dec 10 01:52:00.956: %SYS-5-CONFIG_P: Configured programmatically by process PnP Agent Discovery from console as vty0
*Dec 10 01:52:02.614: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/5, changed state to up
Step 10. View the IP address assigned to the replacement device in the upstream switch that has DHCP server automated by Cisco DNA Center.
SFO-C9300-BN2#sh ip dhcp binding
Bindings from all pools not associated with VRF:
IP address Client-ID/ Lease expiration Type State Interface
Hardware address/
User name
172.20.10.8 0063.6973.636f.2d35. Dec 11 2022 01:51 AM Automatic Active GigabitEthernet1/0/37
3036.312e.6266.6563.
2e30.6634.372d.566c.
31
SFO-C9300-BN2#
Step 11. Under the Unclaimed tab, the replacement 9300 should show up upon Plug and Play. Select the device and click Next.
Step 12. Select Now. Click Next to start the replacement.
Step 13. After reviewing the serial numbers of the devices, click Replace.
Step 14. Click Monitor Replacement Status to view the progress.
Step 15. In the Mark for Replacement focus screen, the device should show the In-Progress status. Click on the In-Progress link to view additional details.
Step 16. The Replace Status tab should show each task turning green as it completes successfully. Click Refresh as needed. Close the pop-up window when all the tasks have been completed.
Note: During RMA Fabric workflow, the step "Refresh replacement device in fabric" is done to just refresh Fabric topology UI post replacement. This will not affect the device replacement process and in this case also you would have seen device replaced successfully. You might encounter an message: "Error:Fabric Refresh". This messaged was handled as part of CSCwd30590 and is available in 2.3.5.3 and beyond.
Step 17. Close the pop-up window when all the tasks have been completed.
Step 18. The replacement is now showing up in the inventory with the new serial number.
Fabric Edge Component Replacement
Catalyst 9400
Topology
Supervisor Replacement
Option I: Manual Method
Step 1. In the Inventory screen, verify that the 9400 is Unreachable. If it is still reachable and the Supervisor needs to be replaced, shut down the switch or disconnect the 9400 uplink cables and resync to put it in the Unreachable state.
Step 2. Click View Device Details.
Step 3. Click the Hardware & Software side menu. View the serial number. Note that this is the serial number of the chassis. It will not change after the supervisor replacement.
Step 4. Click Configuration. Click Export CLI Output. Save the configuration to the local computer.
Note: The configuration exported will have all the passwords and keys masked. We will need to edit the exported configuration file and input the actual passwords.
Step 5. Use a text editor to edit the config file. Remove all the crypto sections in the config.
Step 6. During the config editing, fill in all the keys and passwords that are masked. Save the edited file.
crytpo pki xxxxxxx
enrollment pkcs12
revocation-check crl
!
crytpo pki xxxxxxx
enrollment selfsigned
subject-name xxxxxxxx
revocation-check none
rsakeypair TP-self-signed- xxxxxxxx
!
crytpo pki xxxxxxx
enrollment mode ra
enrollment terminal
usage ssl-client
revocation-check crl none
source interface Loopback0
!
crytpo pki xxxxxxx
enrollment url http://172.16.0.70:80/ejbca/publicweb/apply/scep/sdnscep
fqdn Switch-172-20-33-70
subject-name xxxxxxxx
revocation-check crl
source interface Loopback0
rsakeypair xxxxxxxx
auto-enroll 80 regenerate
!
!
crypto pki certificate chain xxxxxxx
certificate ca xxxxxxxx
xxxx xxxxx xxxx xxxx xxxx
xxxx xxxxx xxxx xxxx xxxx
quit
crypto pki certificate chain xxxxxxx
certificate self-signed xxxxxxxx
xxxx xxxxx xxxx xxxx xxxx
xxxx xxxxx xxxx xxxx xxxx
quit
crypto pki certificate chain xxxxxxx
certificate ca xxxxxxxx
xxxx xxxxx xxxx xxxx xxxx
xxxx xxxxx xxxx xxxx xxxx
quit
crypto pki certificate chain xxxxxxx
xxxx xxxxx xxxx xxxx xxxx
xxxx xxxxx xxxx xxxx xxxx
quit
certificate ca xxxxxxxx
xxxx xxxxx xxxx xxxx xxxx
xxxx xxxxx xxxx xxxx xxxx
quit
aaa server radius dynamic-author
client 172.16.0.68 server-key 7 xxxxxxxx
client 10.122.20.107 server-key 7 xxxxxxxx
enable secret 9 xxxxxxxx
enable secret 9 $9$pU1BrIwOcB9ZgU$QchaDwJB/UXPTDD43w0c4aVxntidT0sU8zP34AbRniM
username sdaadmin privilege 15 secret xxxxxx
snmp-server community xxxxxxxx RO
snmp-server host 172.16.0.70 version 2c xxxxxxxx
radius server dnac-radius_172.16.0.68
address ipv4 172.16.0.68 auth-port 1812 acct-port 1813
timeout 6
retransmit 3
automate-tester username dummy ignore-acct-port probe-on
pac key 7 xxxxxxxx
Step 7. Prepare the new Supervisor in a spare chassis. If the existing chassis can be used during production, prepare the new Supervisor in the existing chassis. Factory default the new Supervisor and copy the edited configuration file into the running config and startup config.
Switch-172-20-33-70#pnpa service reset
PnP Reset command will erase config and reload the device.
Are you sure you want to continue? [yes/no]: y
000214: *Feb 2 13:50:44.858: %PNP-6-PNP_PROFILE_DELETED: PnP profile (pnp-zero-touch) deleted (1/10) by (pid=177, pname=Exec, time=13:50:44 UTC Thu Feb 2 2023)
000215: *Feb 2 13:50:44.862: %SYS-5-CONFIG_P: Configured programmatically by process Exec from console as vty0
Autoinstall will terminate if any input is detected on console
--- System Configuration Dialog ---
Would you like to enter the initial configuration dialog? [yes/no]: no
The enable secret is a password used to protect
access to privileged EXEC and configuration modes.
This password, after entered, becomes encrypted in
the configuration.
-------------------------------------------------
secret should be of minimum 10 characters and maximum 32 characters with
at least 1 upper case, 1 lower case, 1 digit and
should not contain [cisco]
-------------------------------------------------
Enter enable secret: ************
Confirm enable secret: ************
The following configuration command script was created:
enable secret 9 $9$/iz94USZdgM.A.$XSleywnVRgIzQGwWhuzqSfOT2AH8p5blHwyrbKkdfr2
!
end
[0] Go to the IOS command prompt without saving this config.
[1] Return back to the setup without saving this config.
[2] Save this configuration to nvram and exit.
Enter your selection [2]: 0
% You can enter the setup, by typing setup at IOS command prompt
Press RETURN to get started!
Switch>en
Switch#config t
Enter configuration commands, one per line. End with CNTL/Z.
Switch(config)#
Step 8. With the replacement Supervisor in the original chassis, power on the switch. Initiate a sync from Cisco DNA Center to ensure that it is reachable.
Step 9. The Catalyst 9400 will resume its original functions, including its Fabric roles.
Step 10. In the 9400 console, “show inventory” will show the new Supervisor. However, the serial number of the chassis remains the same, which is tracked Cisco DNA Center for this switch.
SFO-C9400-E3#sh inventory
NAME: "Chassis", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis"
PID: C9404R , VID: V01 , SN: FXS2426Q1K8
NAME: "Slot 1 Linecard", DESCR: "Cisco Catalyst 9400 Series 48-Port UPOE w/ 24p mGig 24p RJ-45"
PID: C9400-LC-48UX , VID: V02 , SN: JAD24280NY9
NAME: "Slot 2 Supervisor", DESCR: "Cisco Catalyst 9400 Series Supervisor 1 XL Module"
PID: C9400-SUP-1XL , VID: V02 , SN: JAD24310B2X
NAME: "TenGigabitEthernet2/0/1", DESCR: "GE T"
PID: SP7041-E , VID: , SN: MTC173404PQ
NAME: "Power Supply Module 4", DESCR: "Cisco Catalyst 9400 Series 3200W AC Power Supply"
PID: C9400-PWR-3200AC , VID: V02 , SN: ART2419FCWN
NAME: "Fan Tray", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis Fan Tray"
PID: C9404-FAN , VID: V01 , SN: FXS2426Q12M
Option II: RMA Workflow Method
Step 1. As a background, the serial number of the Supervisor to be replaced is JAD24310B2X. The replacement Supervisor is JAD24310B6M.
SFO-C9400-E3#sh inventory
NAME: "Chassis", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis"
PID: C9404R , VID: V01 , SN: FXS2426Q1K8
NAME: "Slot 1 Linecard", DESCR: "Cisco Catalyst 9400 Series 48-Port UPOE w/ 24p mGig 24p RJ-45"
PID: C9400-LC-48UX , VID: V02 , SN: JAD24280NY9
NAME: "Slot 2 Supervisor", DESCR: "Cisco Catalyst 9400 Series Supervisor 1 XL Module"
PID: C9400-SUP-1XL , VID: V02 , SN: JAD24310B2X
NAME: "TenGigabitEthernet2/0/1", DESCR: "GE T"
PID: SP7041-E , VID: , SN: MTC173404PQ
NAME: "Power Supply Module 4", DESCR: "Cisco Catalyst 9400 Series 3200W AC Power Supply"
PID: C9400-PWR-3200AC , VID: V02 , SN: ART2419FCWN
NAME: "Fan Tray", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis Fan Tray"
PID: C9404-FAN , VID
Step 2. Shut down the 9400. Remove the faulty Supervisor. In the Inventory screen, verify that the 9400 is Unreachable. In the Cisco DNA Center, resync if needed to put it in the Unreachable state.
Step 3. Insert the new Supervisor into the chassis. Ensure that the power is on, and the connections remain as it used to be.
Step 4. Use a console connection to connect to the new Supervisor. It should not have any other existing configurations other than enable password, local username and password, SNMP community strings, VTY lines to enable login, and Interface IP addresses and routing to reach Cisco DNA Center. Verify that Cisco DNA Center can ping the IP address. The config below is for reference.
Note: It is possible to copy the device’s existing physical upstream interface config and routing configure to establish IP connectivity for this device. The configuration below leverage the existing configs.
enable password rma
!
no aaa new-model
!
ip domain name sdn-lab.cisco.com
!
no device-tracking logging theft
!
license boot level network-advantage addon dna-advantage
!
username sdaadmin privilege 15 password 0 rma
!
interface TenGigabitEthernet2/0/1
description Fabric Physical Link
no switchport
dampening
ip address 172.20.10.9 255.255.255.254
no ip redirects
ip pim sparse-mode
ip router isis
load-interval 30
bfd interval 250 min_rx 250 multiplier 3
clns mtu 1400
isis network point-to-point
!
router isis
net 49.0000.aa5b.9acb.1def.00
is-type level-2-only
metric-style wide
log-adjacency-changes
nsf ietf
bfd all-interfaces
!
snmp-server community public RO sdaread
snmp-server community private RO sdawrite
!
line con 0
stopbits 1
line vty 0 4
login local
transport input all
line vty 5 15
login local
transport input all
line vty 16 31
login
transport input ssh
!
netconf-yang
!
ip routing
Step 5. In Inventory, Click Add Device. Select Network Device as Type. Add the IP address that was configured in the new Supervisor. Select the CLI and SNMP credential. Click Add.
Step 6. The new device with the default name should show up in Inventory. Sync the device to ensure that it is Reachable and Managed.
Step 7. Select the existing 9400 that is Unreachable. Click Actions Device Replacement Mark for Replacement.
Step 8. Click Mark to acknowledge the operation.
Step 9. In Device Replacement focus, the device will be shown as Network Readiness Requested. Wait till the device is in Ready for Replacement state.
Step 10. Click on Ready for Replacement to view additional details.
Step 11. Click Actions Device Replacement Replace Device.
Step 12. Review the serial number of the 9400 (note that the serial number is not changed as it is the chassis’ serial number). Click on Managed tab. Select the switch with the default name. Click Next.
Step 13. Select Now. Click Next to start the replacement.
Step 14. After reviewing the serial numbers of the devices, click Replace.
Step 15. Click Monitor Replacement Status to view the progress.
Step 16. In the Device Replacement focus screen, the device should show the In-Progress status. Click on the In-Progress link to view additional details.
Step 17. The Replace Status tab should show each task turning green as it completes successfully. Click Refresh as needed. Close the pop-up window when all the tasks have been completed.
Step 18. The once Unreachable 9400 is now showing in the inventory as Reachable and Managed.
Line Card Replacement
Step 1. As a background, the line card to be replaced has a serial number of JAD2428ONY9. There are static port configurations in one of the ports of this line card.
SFO-C9400-E3#sh inven
NAME: "Chassis", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis"
PID: C9404R , VID: V01 , SN: FXS2416Q1VF
NAME: "Slot 1 Linecard", DESCR: "Cisco Catalyst 9400 Series 48-Port UPOE w/ 24p mGig 24p RJ-45"
PID: C9400-LC-48UX , VID: V02 , SN: JAD24280NY9
NAME: "Slot 2 Supervisor", DESCR: "Cisco Catalyst 9400 Series Supervisor 1 XL Module"
PID: C9400-SUP-1XL , VID: V02 , SN: JAD24310B6M
NAME: "TenGigabitEthernet2/0/1", DESCR: "GE T"
PID: SP7041-E , VID: , SN: MTC18050171
NAME: "Power Supply Module 4", DESCR: "Cisco Catalyst 9400 Series 3200W AC Power Supply"
PID: C9400-PWR-3200AC , VID: V02 , SN: ART2419FCTF
NAME: "Fan Tray", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis Fan Tray"
PID: C9404-FAN , VID: V01 , SN: FXS2417Q314
SFO-C9400-E3#sh start | s 1/0/4
interface GigabitEthernet1/0/4
switchport access vlan 2003
switchport mode access
device-tracking attach-policy IPDT_POLICY
load-interval 30
access-session inherit disable interface-template-sticky
access-session inherit disable autoconf
no macro auto processing
spanning-tree portfast
spanning-tree bpduguard enable
Step 2. Go to Provision > Fabric Sites > (Site) > Host Onboarding > Port Assignment. Select the switch of the interest. Observe the static port assignment of G1/0/4.
Step 3. Go to Provision > Inventory. Click on the switch of the interest. Select Device Details. Click Configuration on the left menu. Observe the configuration on G1/0/4.
Step 4. Remove the existing line card. Put the new line card into the same slot.
Step 5. Verify that the new line card with the new serial number is in the chassis, and that the static port config is there.
SFO-C9400-E3#sh inven
NAME: "Chassis", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis"
PID: C9404R , VID: V01 , SN: FXS2416Q1VF
NAME: "Slot 1 Linecard", DESCR: "Cisco Catalyst 9400 Series 48-Port UPOE w/ 24p mGig 24p RJ-45"
PID: C9400-LC-48UX , VID: V02 , SN: JAD24280LHV
NAME: "Slot 2 Supervisor", DESCR: "Cisco Catalyst 9400 Series Supervisor 1 XL Module"
PID: C9400-SUP-1XL , VID: V02 , SN: JAD24310B6M
NAME: "TenGigabitEthernet2/0/1", DESCR: "GE T"
PID: SP7041-E , VID: , SN: MTC18050171
NAME: "Power Supply Module 4", DESCR: "Cisco Catalyst 9400 Series 3200W AC Power Supply"
PID: C9400-PWR-3200AC , VID: V02 , SN: ART2419FCTF
NAME: "Fan Tray", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis Fan Tray"
PID: C9404-FAN , VID: V01 , SN: FXS2417Q314
SFO-C9400-E3#sh start | s GigabitEthernet1/0/4
interface GigabitEthernet1/0/4
switchport access vlan 2003
switchport mode access
device-tracking attach-policy IPDT_POLICY
load-interval 30
access-session inherit disable interface-template-sticky
access-session inherit disable autoconf
no macro auto processing
spanning-tree portfast
spanning-tree bpduguard enable
Step 6. Verify that the 9400 switch is still reachable.
Step 7. Verify in the Cisco DNA Center GUI that the static port config is there.
Chassis Replacement
Step 1. In Inventory, click on the name of the 9400 to have the chassis replaced. Click on View Details.
Step 2. Click Hardware & Software on the side menu. Note the serial number of the chassis.
Step 3. Power down the C9400. Resync this device if needed until it is shown as Unreachable.
Step 4. In Inventory, click Hardware & Software on the side menu. Note the serial number of the chassis.
Step 5. Select the C9400, click Actions > Device Replacement > Mark for Replacement.
Step 6. Click Mark to acknowledge the operation.
Step 7. In Device Replacement focus, the device will show as Network Readiness Requested.
Step 8. Click on Network Readiness Requested, replacement status is shown.
Step 9. Wait till the status turns to Ready for replacement.
Step 10. Check the config on the neighboring devices, which is the one of Borders. It can be seen that DHCP is configured.
SFO-C9300-BN2#sh run | s dhcp
ip dhcp excluded-address 172.20.10.1 172.20.10.8
ip dhcp excluded-address 172.20.10.10 172.20.10.255
ip dhcp pool GigabitEthernet1/0/40
network 172.20.10.0 255.255.255.0
option 43 ascii 5A1D;B2;K4;I172.16.0.70;J80;
default-router 172.20.10.8
class ciscopnp
address range 172.20.10.1 172.20.10.254
ip dhcp class ciscopnp
option 60 ^ciscopnp
class-map match-any system-cpp-police-dhcp-snooping
description DHCP snooping
ip address dhcp
snmp-server enable traps dhcp
Step 11. Replace the chassis. Put all the original line cards and the Supervisor module back to their original slots.
Step 12. Start the PnP process using the console connection on the Supervisor.
SFO-C9400-E3#pnp service reset
PnP Reset command will erase config and reload the device.
Are you sure you want to continue? [yes/no]: yes
Step 13. The upstream Border should show that the C9400 got an IP address from the DHCP server.
SFO-C9300-BN2#sh ip dhcp bin
Bindings from all pools not associated with VRF:
IP address Client-ID/ Lease expiration Type State Interface
Hardware address/
User name
172.20.10.9 0046.5853.3234.3236. Feb 07 2023 05:29 PM Automatic Active GigabitEthernet1/0/40
5131.4b38
SFO-C9300-BN2#
Step 14. In Device Replacement focus, select the C9400 to be replaced. Click Actions > Device Replacement > Replace Device.
Step 15. In the Unclaimed tab, select serial number of the device that was just onboarded. Click Next.
Step 16. Select Now. Click Next to start the replacement.
Step 17. After verifying that the replacement serial number belongs to the new chassis that has just been put in place, click Replace.
Step 18. Click Monitor Replacement Status to view the progress.
Step 19. In the Device Replacement focus screen, the device should show the In-Progress status. Click on the In-Progress link to view additional details.
Step 20. The Replace Status tab should show each task turning green as it completes successfully. Click Refresh as needed. Close the pop-up window when all the tasks have been completed.
Step 21. The Inventory page should show that new serial number of the replacement chassis is associated with the C9400.
Step 22. The Inventory page should show the C9400 Reachable and Managed.
Fully Loaded Chassis Replacement
Step 1. Review the chassis and the hardware components to be replaced.
SFO-C9400-E3#sh inventory
NAME: "Chassis", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis"
PID: C9404R , VID: V01 , SN: FXS2416Q1VF
NAME: "Slot 1 Linecard", DESCR: "Cisco Catalyst 9400 Series 48-Port UPOE w/ 24p mGig 24p RJ-45"
PID: C9400-LC-48UX , VID: V02 , SN: JAD24280LHV
NAME: "Slot 2 Supervisor", DESCR: "Cisco Catalyst 9400 Series Supervisor 1 XL Module"
PID: C9400-SUP-1XL , VID: V02 , SN: JAD24310B6M
NAME: "TenGigabitEthernet2/0/1", DESCR: "GE T"
PID: SP7041-E , VID: , SN: MTC18050171
NAME: "Power Supply Module 4", DESCR: "Cisco Catalyst 9400 Series 3200W AC Power Supply"
PID: C9400-PWR-3200AC , VID: V02 , SN: ART2419FCTF
NAME: "Fan Tray", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis Fan Tray"
PID: C9404-FAN , VID: V01 , SN: FXS2417Q314
Replacement chassis
SFO-C9400-E3#sh inven
NAME: "Chassis", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis"
PID: C9404R , VID: V01 , SN: FXS2426Q1K8
NAME: "Slot 1 Linecard", DESCR: "Cisco Catalyst 9400 Series 48-Port UPOE w/ 24p mGig 24p RJ-45"
PID: C9400-LC-48UX , VID: V02 , SN: JAD24280NY9
NAME: "Slot 2 Supervisor", DESCR: "Cisco Catalyst 9400 Series Supervisor 1 XL Module"
PID: C9400-SUP-1XL , VID: V02 , SN: JAD24310B2X
NAME: "TenGigabitEthernet2/0/1", DESCR: "GE T"
PID: SP7041-E , VID: , SN: MTC173404PQ
NAME: "Power Supply Module 4", DESCR: "Cisco Catalyst 9400 Series 3200W AC Power Supply"
PID: C9400-PWR-3200AC , VID: V02 , SN: ART2419FCWN
NAME: "Fan Tray", DESCR: "Cisco Catalyst 9400 Series 4 Slot Chassis Fan Tray"
PID: C9404-FAN , VID: V01 , SN: FXS2426Q12M
Step 2. Power remove the faulty C9400 from the network. Resync this device if needed until it is shown as Unreachable.
Step 3. In Inventory, click Hardware & Software on the side menu.
Step 4. Note the serial number of the chassis.
Step 5. Select the C9400, click Actions > Device Replacement > Mark for Replacement.
Step 6. Click Mark to acknowledge the operation.
Step 7. In Device Replacement focus, the device will show as Network Readiness Requested.
Step 8. Wait till the status changed to Ready for Replacement.
Step 9. Connect the fully loaded replacement c9400 chassis to the network.
Step 10. Start the PnP process using the console connection on the Supervisor. Monitor the console outputs to ensure that the PnP proceed successfully.
switch#pnp service reset
PnP Reset command will erase config and reload the device.
Are you sure you want to continue? [yes/no]: yes
Step 11. In Device Replacement focus, select the C9400 to be replaced. Click Actions > Device Replacement > Replace Device.
Step 12. In the Unclaimed tab, select serial number of the device that was just onboarded. Click Next.
Step 13. Select Now. Click Next to start the replacement.
Step 14. After verifying that the replacement serial number belongs to the new chassis that has just been put in place, click Replace.
Step 15. Click Monitor Replacement Status to view the progress.
Step 16. In the Device Replacement focus screen, the device should show the In-Progress status. Click on the In-Progress link to view additional details.
Step 17. The Replace Status tab should show each task turning green as it completes successfully. Click Refresh as needed. Close the pop-up window when all the tasks have been completed.
Step 18. The Inventory page should show that new serial number of the replacement chassis is associated with the C9400.
Border/Control Plane Replacement
Catalyst 9300
Topology
Procedures
Step 1. Select the device to be replaced. Click Actions > Device Replacement > Mark for Replacement.
Step 2. Click Mark to acknowledge the operation.
Step 3. In Device Replacement focus, the device will show as Network Readiness Requested.
Step 4. Click on Network Readiness Requested, replacement status is shown.
Step 5. Wait till the device is in Ready for Replacement status.
Step 6. Click on Ready for Replacement, replacement status shows that DHCP server has been configured in the neighboring Border switch.
Step 7. It can be observed that a DHCP server is configured by Cisco DNA Center in the upstream switch.
SFO-C9300-BN2#sh run | s dhcp
ip dhcp excluded-address 172.20.10.1 172.20.10.10
ip dhcp excluded-address 172.20.10.12 172.20.10.255
ip dhcp pool GigabitEthernet1/0/47
network 172.20.10.0 255.255.255.0
option 43 ascii 5A1D;B2;K4;I172.16.0.70;J80;
default-router 172.20.10.10
class ciscopnp
address range 172.20.10.1 172.20.10.254
ip dhcp class ciscopnp
option 60 ^ciscopnp
class-map match-any system-cpp-police-dhcp-snooping
description DHCP snooping
ip address dhcp
snmp-server enable traps dhcp
SFO-C9300-BN2#
Step 8. Type in “pnp service reset” in the replacement device to trigger PnP for a 9300 that is on and after IOS-XE 16.12. For a release prior to 16.12, refer to https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/tech_notes/b_dnac_sda_lan_automation_deployment.html for the commands to use.
SFO-C9300-BN2#pnp service reset
PnP Reset command will erase config and reload the device.
Are you sure you want to continue? [yes/no]: y
Step 9. In the Device Replacement focus, select the Border switch to be replaced, go to Actions > Device Replacement > Replace Device.
Step 10. Under the Unclaimed tab, the replacement 9300 should show up upon Plug and Play. Select the device and click Next.
Step 11. Select Now. Click Next to start the replacement.
Step 12. After reviewing the serial numbers of the devices, click Replace.
Step 13. Click Monitor Replacement Status to view the progress.
Step 14. In the Device Replacement focus screen, the device should show the In-Progress status. Click on the In-Progress link to view additional details.
Step 15. The Replace Status tab should show each task turning green as it completes successfully. Click Refresh as needed. Close the pop-up window when all the tasks have been completed.
Step 16. Close the pop-up window when all the tasks have been completed. If this pop-up window was closed previously, clicking the fresh button when it turns yellow will show the latest status of the replacement.
Step 17. The replacement is now showing up in the inventory with the new serial number.
Switch Stack Member Replacement
Catalyst 9300 Switch Stack
Topology
Procedures
Step 1. In the Inventory screen, view the models and serial numbers with the Fabric in a Box switch members.
Step 2. Click on the FIAB name, click View Device Details.
Step 3. Click on Hardware & Software side menu. Similar information will show with the serial numbers corresponding to specific model numbers of the switch stack members.
Step 4. Type in “show switch” to view which switch is the active switch, and the priority of the switch members.
DFW-C9300-FIAB#sh switch
Switch/Stack Mac Address : 7001.b59f.8e00 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
-------------------------------------------------------------------------------------
*1 Active 7001.b59f.8e00 1 V01 Ready
2 Standby 7001.b5a0.1080 1 V01 Ready
DFW-C9300-FIAB#show inventory
NAME: "c93xx Stack", DESCR: "c93xx Stack"
PID: C9300-24UX , VID: V01 , SN: FCW2214G0AN
NAME: "Switch 1", DESCR: "C9300-24UX"
PID: C9300-24UX , VID: V01 , SN: FCW2214G0AN
NAME: "StackPort1/1", DESCR: "StackPort1/1"
PID: STACK-T1-50CM , VID: V01 , SN: LCC2201G59W
NAME: "StackPort1/2", DESCR: "StackPort1/2"
PID: STACK-T1-50CM , VID: V01 , SN: LCC2201G59G
NAME: "Switch 1 - Power Supply B", DESCR: "Switch 1 - Power Supply B"
PID: PWR-C1-1100WAC , VID: V02 , SN: DTN2152V2G7
NAME: "Switch 1 FRU Uplink Module 1", DESCR: "8x10G Uplink Module"
PID: C9300-NM-8X , VID: V01 , SN: FOC22125DNT
NAME: "Te1/1/1", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V02 , SN: AVC2334000M
NAME: "Te1/1/7", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V01 , SN: AVC211520X5
NAME: "Te1/1/8", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V01 , SN: AVC1952266S
NAME: "Switch 2", DESCR: "C9300-48U"
PID: C9300-48U , VID: V01 , SN: FOC2214Q0RM
NAME: "StackPort2/1", DESCR: "StackPort2/1"
PID: STACK-T1-50CM , VID: V01 , SN: LCC2201G59G
NAME: "StackPort2/2", DESCR: "StackPort2/2"
PID: STACK-T1-50CM , VID: V01 , SN: LCC2201G59W
NAME: "Switch 2 - Power Supply B", DESCR: "Switch 2 - Power Supply B"
PID: PWR-C1-1100WAC , VID: V02 , SN: DTN2152V2L4
NAME: "Switch 2 FRU Uplink Module 1", DESCR: "8x10G Uplink Module"
PID: C9300-NM-8X , VID: V01 , SN: FOC22125C87
NAME: "Te2/1/1", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V02 , SN: AVC23330067
NAME: "Te2/1/2", DESCR: "10/100/1000BaseTX SFP"
PID: , VID: , SN: MTC13040227
NAME: "Te2/1/3", DESCR: "SFP-10GBase-SR"
PID: SFP-10G-SR , VID: V02 , SN: AGD131330CA
NAME: "Te2/1/4", DESCR: "SFP-10GBase-SR"
PID: SFP-10G-SR , VID: V03 , SN: FNS1703007B
NAME: "Te2/1/7", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V01 , SN: AVC2001200B
NAME: "Te2/1/8", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V01 , SN: AVC222620WE
DFW-C9300-FIAB#
Note: It is suggested to make the switch member to be replaced no-active switch so as to not let the replacement switch overwrite the other switch members’ configs.
Step 5. Type in “switch x priority” to hard set the switch priority to make the switch member to be replaced no active. Higher priority makes it more likely to become the active switch.
Step 6. Reload the switch stack to make the change take effect.
Note: This is assumed that the currently Active switch needs to be replaced. Even if the switch to be replaced is not the Active switch, it is still suggested to set one of the remaining switch to be Active so it can win over the incoming replacement switch.
DFW-C9300-FIAB#switch 2 priority 15
DFW-C9300-FIAB#switch 1 priority 14
DFW-C9300-FIAB#sh switch
Switch/Stack Mac Address : 7001.b59f.8e00 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
-------------------------------------------------------------------------------------
*1 Active 7001.b59f.8e00 14 V01 Ready
2 Standby 7001.b5a0.1080 15 V01 Ready
DFW-C9300-FIAB#
DFW-C9300-FIAB#reload
Reload command is being issued on Active unit, this will reload the whole stack
Proceed with reload? [confirm]
Jan 9 22:06:04.240: %SYS-5-RELOAD: Reload requested by sdaadmin on console. Reload Reason: Reload Command.
Step 7. After the reload is completed, type in “show switch” to verify that the change takes effect for the switch stack.
DFW-C9300-FIAB#show switch
Switch/Stack Mac Address : 7001.b5a0.1080 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
-------------------------------------------------------------------------------------
1 Member 7001.b59f.8e00 14 V01 Ready
*2 Active 7001.b5a0.1080 15 V01 Ready
Step 8. On the replacement switch member, make sure that it has a lower switch priority.
switch#switch 1 priority 14
switch#sh switch
Switch/Stack Mac Address : 7061.7b4f.db80 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
-------------------------------------------------------------------------------------
*1 Active 7061.7b4f.db80 14 V03 Ready
Step 9. Remove the defective switch member, connect the new switch member to the stack.
Step 10. Resync the FIAB in the inventory.
Step 11. Click on the name of the switch stack.
Step 12. Click on Hardware & Software. Observe that the serial number of the switch member is changed.
Step 13. On the switch console, verify the switch member changed.
DFW-C9300-FIAB#sh switch
Switch/Stack Mac Address : 7001.b5a0.1080 - Local Mac Address
Mac persistency wait time: Indefinite
H/W Current
Switch# Role Mac Address Priority Version State
-------------------------------------------------------------------------------------
1 Member 7001.b5ee.e080 14 V01 Ready
*2 Active 7001.b5a0.1080 15 V01 Ready
DFW-C9300-FIAB#sh inventory
NAME: "c93xx Stack", DESCR: "c93xx Stack"
PID: C9300-48U , VID: V01 , SN: FOC2214Q0RM
NAME: "Switch 1", DESCR: "C9300-24UX"
PID: C9300-24UX , VID: V01 , SN: FOC2214Z00J
NAME: "StackPort1/1", DESCR: "StackPort1/1"
PID: STACK-T1-50CM , VID: V01 , SN: LCC2201G59W
NAME: "StackPort1/2", DESCR: "StackPort1/2"
PID: STACK-T1-50CM , VID: V01 , SN: LCC2201G59G
NAME: "Switch 1 - Power Supply B", DESCR: "Switch 1 - Power Supply B"
PID: PWR-C1-1100WAC , VID: V02 , SN: DTN2152V2G9
NAME: "Switch 2", DESCR: "C9300-48U"
PID: C9300-48U , VID: V01 , SN: FOC2214Q0RM
NAME: "StackPort2/1", DESCR: "StackPort2/1"
PID: STACK-T1-50CM , VID: V01 , SN: LCC2201G59G
NAME: "StackPort2/2", DESCR: "StackPort2/2"
PID: STACK-T1-50CM , VID: V01 , SN: LCC2201G59W
NAME: "Switch 2 - Power Supply B", DESCR: "Switch 2 - Power Supply B"
PID: PWR-C1-1100WAC , VID: V02 , SN: DTN2152V2L4
NAME: "Switch 2 FRU Uplink Module 1", DESCR: "8x10G Uplink Module"
PID: C9300-NM-8X , VID: V01 , SN: FOC22125C87
NAME: "Te2/1/1", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V02 , SN: AVC23330067
NAME: "Te2/1/2", DESCR: "10/100/1000BaseTX SFP"
PID: , VID: , SN: MTC13040227
NAME: "Te2/1/3", DESCR: "SFP-10GBase-SR"
PID: SFP-10G-SR , VID: V02 , SN: AGD131330CA
NAME: "Te2/1/4", DESCR: "SFP-10GBase-SR"
PID: SFP-10G-SR , VID: V03 , SN: FNS1703007B
NAME: "Te2/1/7", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V01 , SN: AVC2001200B
NAME: "Te2/1/8", DESCR: "10/100/1000BaseTX SFP"
PID: GLC-TE , VID: V01 , SN: AVC222620WE
DFW-C9300-FIAB#
Fusion Replacement
Catalyst 9300
Topology
Procedures
Step 1. Select the device to be replaced. Click Actions > Device Replacement > Mark for Replacement.
Step 2. Click Mark to acknowledge the operation.
Note: There would be use cases that the device to be replaced is reachable through IP. It is suggested to show down this device or disconnect the connection to the device and then resync in Cisco DNA Center to make this device in Unreachable state. This would ensure that the RMA workflow will proceed successfully.
Step 3. In Device Replacement focus, wait till the device is shown as Ready for Replacement state.
Step 4. Click on Ready for Replacement to view additional details.
Step 5. Prepare the basic configuration on the replacement device so it can be discovered by Cisco DNA Center. Note that interface GigabitEther1/0/8 is used to be connected to the other Fusion router for the discovery.
enable password rma
!
no aaa new-model
!
ip domain name sdn-lab.cisco.com
!
no device-tracking logging theft
!
license boot level network-advantage addon dna-advantage
!
username sdaadmin privilege 15 password 0 rma
!
interface Loopback0
ip address 172.20.32.3 255.255.255.255
!
interface GigabitEthernet1/0/6
no switchport
ip address 172.20.32.166 255.255.255.252
!
interface GigabitEthernet1/0/8
no switchport
ip address 172.20.32.170 255.255.255.252
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip route 0.0.0.0 0.0.0.0 172.20.32.169
!
snmp-server community public RO sdaread
snmp-server community private RO sdawrite
!
line con 0
stopbits 1
line vty 0 4
login local
transport input all
line vty 5 15
login local
transport input all
line vty 16 31
login
transport input ssh
!
netconf-yang
ip routing
router eigrp 100
network 172.20.32.168 0.0.0.3
network 172.20.32.164 0.0.0.3
network 172.20.32.3 0.0.0.0
no passive interface g1/0/8
Step 6. The neighboring Fusion router is configured with EIGRP to advertise the route towards the replacement device.
SFO-3800-Fusion1#sh run | s router eigrp
router eigrp 100
network 172.20.32.168 0.0.0.3
no passive-interface g1/1/2
interface GigabitEthernet1/1/2
no switchport
ip address 172.20.32.169 255.255.255.252no passive interface g1/0/8
Step 7. Go to the main menu and click Tools > Discovery.
Step 8. Click Add Discovery.
Step 9. Click Next.
Step 10. Type in the discovery name. Select IP Address Range. Type in the loopback address of the Fusion to be replaced. Clicked Next.
Step 11. Select the CLI credential. Click Next.
Step 12. Select the SNMP Read community. Clicked Next.
Step 13. Select the SNMP Write community. Clicked Next.
Step 14. Click Next.
Step 15. Select Skip site assignment for now. Click Next.
Step 16. Clicked Next.
Step 17. Click Start Discovery and Telemetry.
Step 18. Clicked View Discovery.
Step 19. The discovery job should have been completed successfully.
Step 20. View the Inventory focus, the new device will show up in the inventory.
Step 21. Click Actions Device Replacement > Replace Device.
Step 22. Review the serial number of the Fusion to be replaced. Click on Managed tab. Select the replacement Fusion. Click Next.
Step 23. Select Now. Click Next to start the replacement.
Step 24. After reviewing the serial numbers of the devices, click Replace.
Step 25. Click Monitor Replacement Status to view the progress.
Step 26. In the Device Replacement focus screen, the device should be in In-Progress status. Click on the In-Progress link to view additional details.
Step 27. The Replace Status tab should show each task turning green as it completes successfully. Click Refresh as needed. Close the pop-up window when all the tasks have been completed.
Step 28. The replacement is now showing up in the inventory with the new serial number.
Appendix: Reference
Cisco DNA Center User Guide, Release 2.3.5
https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-3-5/user_guide/b_cisco_dna_center_ug_2_3_5/m_dnac_workflows.html#Cisco_Task_in_List_GUI.dita_95db7af2-22b9-4802-bf1a-4302b0dc78c5
Cisco DNA Center User Guide, Release 2.3.5 – Replacing a Faulty Device
https://www.cisco.com/c/en/us/td/docs/cloud-systems-management/network-automation-and-management/dna-center/2-3-5/user_guide/b_cisco_dna_center_ug_2_3_5/b_cisco_dna_center_ug_2_3_5_chapter_011.html#id_114181
|
