09-27-2019 01:10 AM
Hello,
We are currently building a test setup with DNA Center and a couple of routers en switches. Now i was in the process of updating the DNA center with latest software step by step. But now one update failed and i have no longer access to the web interface of DNA Center. I still do have access to the maglev CLI on the same ip address via SSH.
The update was stuck for a long time at below:
$ sudo maglev system_updater update_info System update status: Version successfully installed : 1.1.0.784 Version currently processed : 1.2.0.1013 Update phase : Updating the host components Update details : Updating the node 10.0.32.1 [phase 4] Progress : 48% Updater State: Currently processed version : 1.2.0.1013 State : INSTALLING_UPDATES Sub-State : INSTALLED_PREINSTALL_CLUSTER_UPDATES Details : Updating the host components Source : system-updater
Then i saw below:
$ maglev catalog package display maglev-1 [main - https://kong-frontend.maglev-system.svc.cluster.local:443] NAME DISPLAY_NAME VERSION STATE INFO -------------------------------------------------------------------------------------------------------- application-policy Application Policy 2.1.29.170002 PARTIAL Package needs to be pulled/downloaded assurance Assurance - Base 1.2.12.125 PARTIAL Package needs to be pulled/downloaded automation-core NCP - Services 2.1.31.60019 PARTIAL Package needs to be pulled/downloaded base-provision-core Automation - Base 2.1.31.60019 PARTIAL Package needs to be pulled/downloaded command-runner Command Runner 2.1.31.60019 PARTIAL Package needs to be pulled/downloaded device-onboarding Device Onboarding 2.1.31.60019.2 PARTIAL Package needs to be pulled/downloaded dnac-platform DNAC Platform 1.0.12.4 PARTIAL Package needs to be pulled/downloaded icap-automation Automation - Intelligent Capture 2.1.31.60019 PARTIAL Package needs to be pulled/downloaded image-management Image Management 2.1.31.60019 PARTIAL Package needs to be pulled/downloaded ncp-system NCP - Base 2.1.31.60019 PARTIAL Package needs to be pulled/downloaded ndp-base-analytics Network Data Platform - Base Analytics 1.1.11.15 PARTIAL Package needs to be pulled/downloaded ndp-platform Network Data Platform - Core 1.1.11.121 PARTIAL Package needs to be pulled/downloaded ndp-ui Network Data Platform - Manager 1.1.11.12 PARTIAL Package needs to be pulled/downloaded network-visibility Network Controller Platform 2.1.31.60019.2 PARTIAL Package needs to be pulled/downloaded path-trace Path Trace 2.1.31.60019 PARTIAL Package needs to be pulled/downloaded platform-ui Cisco DNA Center UI 1.3.1.248 READY sd-access SD Access 2.1.31.60019.2 PARTIAL Package needs to be pulled/downloaded sensor-assurance Assurance - Sensor 1.2.12.124 PARTIAL Package needs to be pulled/downloaded sensor-automation Automation - Sensor 2.1.31.60019 PARTIAL Package needs to be pulled/downloaded system System 1.2.0.1013 READY
when i check the status of one of the package:
$ maglev catalog package status application-policy:2.1.29.170002 maglev-1 [main - https://kong-frontend.maglev-system.svc.cluster.local:443] KIND RESOURCE STATE MESSAGE ---------------------------------------------------------------------------------------------------------------------- Package application-policy:2.1.29.170002 PARTIAL One or more child resources are not available yet ServiceBundleGroup fusion/application-policy:2.1.29.170002 NOT_FOUND Manifest not available yet ServiceBundleGroup maglev-system/application-policy-ui:2.1.29.170002 NOT_FOUND Manifest not available yet
So i tried to delete them till they were at ready state:
$ maglev catalog package delete assurance:1.2.12.125 ok
After all was deleted
$ maglev catalog package display maglev-1 [main - https://kong-frontend.maglev-system.svc.cluster.local:443] NAME DISPLAY_NAME VERSION STATE INFO -------------------------------------------------------------------------------------------------------- application-policy Application Policy 2.1.28.170011 READY assurance Assurance - Base 1.2.11.304 READY automation-core NCP - Services 2.1.28.60244.9 READY base-provision-core Automation - Base 2.1.28.60244.9 READY command-runner Command Runner 2.1.28.60244 READY device-onboarding Device Onboarding 2.1.28.60244 READY dnac-platform DNAC Platform 1.0.8.8 READY icap-automation Automation - Intelligent Capture 2.1.28.60244 READY image-management Image Management 2.1.28.60244 READY ncp-system NCP - Base 2.1.28.60244 READY ndp-base-analytics Network Data Platform - Base Analytics 1.1.11.8 READY ndp-platform Network Data Platform - Core 1.1.11.77 READY ndp-ui Network Data Platform - Manager 1.1.11.8 READY network-visibility Network Controller Platform 2.1.28.60244.9 READY path-trace Path Trace 2.1.28.60244 READY platform-ui Cisco DNA Center UI 1.3.1.248 READY sd-access SD Access 2.1.28.60244.9 READY sensor-assurance Assurance - Sensor 1.2.10.254 READY sensor-automation Automation - Sensor 2.1.28.60244 READY system System 1.2.0.1013 READY
But about 3 / 4 hours later the update failed.
System update status: Version successfully installed : 1.1.0.784 Version currently processed : 1.2.0.1013 Update phase : failed Update details : Node update took longer to complete in 10.0.32.1 [ version 1.2.0.1013 phase 4 ] Progress : 48% Updater State: Currently processed version : 1.2.0.1013 State : FAILED Sub-State : INSTALLED_HOST_COMPONENTS Details : Node update took longer to complete in 10.0.32.1 [ version 1.2.0.1013 phase 4 ] Source : system-updater
I then tried to force it back to an older version.
$ maglev system_updater update_system 1.1.0.784 -f System update download initiated
But also failed later
$ sudo maglev system_updater update_info System update status: Version successfully installed : 1.1.0.784 Version currently processed : 1.1.0.784 Update phase : failed Update details : Updating node 10.0.32.1 failed Progress : 52% Updater State: Currently processed version : 1.1.0.784 State : FAILED Sub-State : INSTALLED_HOST_COMPONENTS Details : Updating node 10.0.32.1 failed Source : system-updater
Eventhough all packages are deployed
$ maglev package status maglev-1 [main - https://kong-frontend.maglev-system.svc.cluster.local:443] NAME DISPLAY_NAME DEPLOYED AVAILABLE STATUS PROGRESS --------------------------------------------------------------------------------------------------------------------------------------- application-policy Application Policy 2.1.28.170011 2.1.42.170001 DEPLOYED assurance Assurance - Base 1.2.11.304 1.3.1.140 DEPLOYED automation-core NCP - Services 2.1.28.60244.9 2.1.43.60042 DEPLOYED base-provision-core Automation - Base 2.1.28.60244.9 2.1.43.60042 DEPLOYED command-runner Command Runner 2.1.28.60244 2.1.43.60042 DEPLOYED device-onboarding Device Onboarding 2.1.28.60244 2.1.43.60042 DEPLOYED dnac-platform DNAC Platform 1.0.8.8 1.1.1.2 DEPLOYED icap-automation Automation - Intelligent Capture 2.1.28.60244 2.1.43.60042 DEPLOYED image-management Image Management 2.1.28.60244 2.1.43.60042 DEPLOYED ncp-system NCP - Base 2.1.28.60244 2.1.43.60042 DEPLOYED ndp-base-analytics Network Data Platform - Base Analytics 1.1.11.8 1.3.1.204 DEPLOYED ndp-platform Network Data Platform - Core 1.1.11.77 1.3.1.615 DEPLOYED ndp-ui Network Data Platform - Manager 1.1.11.8 1.3.1.185 DEPLOYED network-visibility Network Controller Platform 2.1.28.60244.9 2.1.43.60046 DEPLOYED path-trace Path Trace 2.1.28.60244 2.1.43.60042 DEPLOYED platform-ui Cisco DNA Center UI 1.3.1.248 1.3.1.250 DEPLOYED sd-access SD Access 2.1.28.60244.9 2.1.43.60045 DEPLOYED sensor-assurance Assurance - Sensor 1.2.10.254 1.3.0.1313 DEPLOYED sensor-automation Automation - Sensor 2.1.28.60244 2.1.43.60042 DEPLOYED system System 1.1.0.784 1.2.0.1013 DEPLOYED
State of the packages is ready
$ maglev catalog package display maglev-1 [main - https://kong-frontend.maglev-system.svc.cluster.local:443] NAME DISPLAY_NAME VERSION STATE INFO -------------------------------------------------------------------------------------------------------- application-policy Application Policy 2.1.28.170011 READY assurance Assurance - Base 1.2.11.304 READY automation-core NCP - Services 2.1.28.60244.9 READY base-provision-core Automation - Base 2.1.28.60244.9 READY command-runner Command Runner 2.1.28.60244 READY device-onboarding Device Onboarding 2.1.28.60244 READY dnac-platform DNAC Platform 1.0.8.8 READY icap-automation Automation - Intelligent Capture 2.1.28.60244 READY image-management Image Management 2.1.28.60244 READY ncp-system NCP - Base 2.1.28.60244 READY ndp-base-analytics Network Data Platform - Base Analytics 1.1.11.8 READY ndp-platform Network Data Platform - Core 1.1.11.77 READY ndp-ui Network Data Platform - Manager 1.1.11.8 READY network-visibility Network Controller Platform 2.1.28.60244.9 READY path-trace Path Trace 2.1.28.60244 READY platform-ui Cisco DNA Center UI 1.3.1.248 READY sd-access SD Access 2.1.28.60244.9 READY sensor-assurance Assurance - Sensor 1.2.10.254 READY sensor-automation Automation - Sensor 2.1.28.60244 READY system System 1.2.0.1013 READY
Now my question is if there is something else i can try to restore the DNAC via maglev? I did download the DNAC ISO from cisco website, but could not find the instructions on how to do a re-installation of the DNAC software.
Thanks in advance!
09-27-2019 02:14 AM
Ok, i seem to have access again after disabling maintenance mode via maglev:
$ maglev maintenance disable Disable cluster maintenance mode.
now it seems that the server is trying to restore itself, ill give it a weekend and will see what state it is in Monday.
Automation and Assurance services have been temporarily disrupted. The system is working to restore this functionality. More Info
11-20-2019 07:53 AM
Hi,
Did your issue get resolved after you disabled the maintenance mode?
11-21-2019 12:29 AM
Hi,
After disabling maintenance mode i was able to access the web interface again, but still the system was not working properly and did not restore. I had to reinstall the software on the appliance.
11-21-2019 01:13 PM
Hi,
Did you not wait the 90 seconds they say you must in the upgrade guide?
It doesn't say anywhere how to fix this if the user doesn't wait :(
I have the similar upgrade issue.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: