cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
374
Views
2
Helpful
4
Replies

DNA Center 2.3.3.5 has installed packages for 2.3.7.7

MarkAntony
Level 1
Level 1

Hi Everyone, I recently reimaged my DNA Servers (3-node cluster) to 2.3.3.5 after a failed upgrade, plan is to reimage, update packages as per previous then restore backup. The reimage went well, all ok there, I then went to Software management selected all ensuring all versions matched that to what I previously used for 2.3.3.5 then clicked Install. All was good with packages installing until it got to 'Application Visibility Service' where it froze at 90% and it eventually failed after a couple hours. After investigating the below error was present

DEPLOYMENT_ERROR - Exception in task - Maximum wait time 5400 seconds exceeded for the following services to be ready: network-design-service, spf-service-manager-service, apic-em-network-programmer-service, application-recognition-service

When i checked these other dependent services I found the network-visibility, NCP–BASE & automation-core (NCP – SYSTEM) had all been updated to 2.1.720.61514 which is for 2.3.7.7 and the network-visibility service was not starting because of this. Prior to starting the package installs these services were all correctly on 2.1.515.60238 for 2.3.3.5. Unsure why its even touched these.

I've undeployed network-visibility, but unable to undeploy NCP-BASE & automation core, anyone know how to get these 3x services back to 2.1.515.60238 as I think this will fix the problem. Current Package status output below. Just to also mention that these 3x services on 2.1.515.60238 are now also showing in Software management in GUI as available to install but if I attempt to install I get a warning stating "Warning packages not specified’ then nothing happens. many thanks.

btw I do have a TAC Case logged for this too but waiting next steps.

NAME                      DISPLAY_NAME                             DEPLOYED        AVAILABLE       STATUS                    PROGRESS
---------------------------------------------------------------------------------------------------------------------------------------
access-control-application Access Control Application               2.1.515.60238   2.1.720.61514   DEPLOYED
ai-network-analytics      AI Network Analytics                     2.9.24.406      3.3.58.21       DEPLOYED
app-hosting               Application Hosting                      1.9.02209020733 2.3.12406270628 DEPLOYED
application-policy        Application Policy                       2.1.515.117391  2.1.720.117227  DEPLOYED
application-registry      Application Registry                     2.1.515.117391  2.1.720.117227  DEPLOYED
application-visibility-service Application Visibility Service           2.1.515.117391  2.1.720.117227  DEPLOYMENT_ERROR - Exception in task - Maximum wait time 5400 seconds exceeded for the following services to be ready: network-design-service, spf-service-manager-service, apic-em-network-programmer-service, application-recognition-service
appsvc-remediation        Application and Service Remediation      -               1.1.32          NOT_DEPLOYED
assurance                 Assurance - Base                         2.3.3.463       2.3.7.7006      DEPLOYED
automation-core           NCP - Services                           2.1.720.60128   -               DEPLOYED
base-provision-core       Automation - Base                        2.1.515.60238   2.1.720.61514   DEPLOYED
catalyst-center-api-catalog Catalyst Center API Catalog              -               1.15.69         NOT_DEPLOYED
cisco-dna-docs            Documentation and Help                   -               2.1.720.60128   NOT_DEPLOYED
cloud-connectivity-contextual-content Cloud Connectivity - Contextual Content  2.4.1.338       2.11.2.542      DEPLOYED
cloud-connectivity-data-hub Cloud Connectivity - Data Hub            1.8.43          1.15.57         DEPLOYED
cloud-connectivity-tethering Cloud Connectivity - Tethering           2.30.1.72       2.36.1.47       DEPLOYED
cloud-provision-core      Cloud Device Provisioning Application    2.1.515.60238   2.1.720.60128   DEPLOYED
command-runner            Command Runner                           2.1.515.60238   2.1.720.60128   DEPLOYED
device-onboarding         Device Onboarding                        2.1.515.60238   2.1.720.60128   DEPLOYED
disaster-recovery         Disaster Recovery                        2.1.515.360031  2.1.720.360029  DEPLOYED
dna-core-apps             Network Experience Platform - Core       2.1.515.60238   2.1.720.60128   DEPLOYED
dnac-platform             Cisco DNA Center Platform                1.8.1.137       1.15.105.0      DEPLOYED
dnac-search               Cisco DNA Center Global Search           1.8.1.10        1.15.1.12       DEPLOYED
endpoint-analytics        AI Endpoint Analytics                    1.7.702         1.11.1185       DEPLOYED
group-based-policy-analytics Group-Based Policy Analytics             2.3.3.35        2.3.7.140       DEPLOYED
icap-automation           Automation - Intelligent Capture         2.1.515.60238   2.1.720.60128   DEPLOYED
image-management          Image Management                         2.1.515.60238   2.1.720.61514   DEPLOYED
ise-bridge                Cisco Identity Services Engine Bridge    2.1.515.450     2.1.716.90701   DEPLOYED
machine-reasoning         Machine Reasoning                        2.1.515.210125  2.1.720.210013  DEPLOYED
ncp-system                NCP - Base                               2.1.720.60128   -               DEPLOYED
ndp-base-analytics        Network Data Platform - Base Analytics   1.8.239         2.3.7.750156    DEPLOYED
ndp-platform              Network Data Platform - Core             1.8.396         1.9.5055        DEPLOYED
ndp-ui                    Network Data Platform - Manager          1.8.244         1.9.5001        DEPLOYED
network-visibility        Network Controller Platform              -               2.1.720.61514   NOT_DEPLOYED
path-trace                Path Trace                               2.1.515.60238   2.1.720.61514   DEPLOYED
platform-ui               Cisco Catalyst Center UI                 1.7.1.339       1.7.7.71        DEPLOYED
rbac-extensions           RBAC Extensions                          2.1.515.1900002 2.1.720.1900009 DEPLOYED
rogue-management          Rogue and aWIPS                          2.5.0.20        2.9.0.654       DEPLOYED
sd-access                 SD Access                                2.1.515.60238   2.1.720.61514   DEPLOYED
sensor-assurance          Assurance - Sensor                       2.3.3.375       2.3.7.7003      DEPLOYED
sensor-automation         Automation - Sensor                      2.1.515.60238   2.1.720.60128   DEPLOYED
ssa                       Stealthwatch Security Analytics          2.1.515.1090110 2.1.720.1090100 DEPLOYED
support-services          Support Services                         2.1.510.880029  2.1.720.880010  DEPLOYED
system                    System                                   1.7.769         1.8.232         DEPLOYED
system-commons            System Commons                           2.1.515.60238   2.1.720.61514   DEPLOYED
system-remediation        System Remediation                       -               1.4.0           NOT_DEPLOYED
umbrella                  Cisco Umbrella                           2.1.515.590102  2.1.720.590098  DEPLOYED
wide-area-bonjour         Wide Area Bonjour                        2.4.511.75063   2.4.718.75196   DEPLOYED

 

4 Replies 4

maflesch
Cisco Employee
Cisco Employee

Hi Mark. Yes, please open a TAC case. This issue has been seen before with restoring 2.3.3.5 where it does exactly as you state and updates the latest available package for that service, even when not intended. 

As for removing those packages, that's probably not going to happen. Once those packages updated, it updated the schema inside those services and we can't revert that. It's most likely going to be a reimage again, but we need to identify why this happened first.

MarkAntony
Level 1
Level 1

Thanks @maflesch , it is as you state, after a 2 hour TAC call of them attempting to remove packages etc they advised best to reimage again, this time they asked me just to reimage the 1x node and get that back fully operational first with my backup restored then the other 2x can be reimaged and added to cluster in due course. They couldn't find the cause of the problem.

MarkAntony
Level 1
Level 1

Reimaged 1x node to 2.3.3.5 and after doing a 'maglev package status' all base/default Services are on correct versions as expected but concerning that 'available' is showing non-compatible versions available which look to be for 2.3.7.7, wonder if this the cause of my previous attempt and when i installed the additional packages required it automatically update some of these some being core NCP Services. awaiting to hear back form TAC and holding off installing anything, really dont wont to rebuild for a 3rd time ha

MarkAntony_0-1741960968367.png

 

Hi,
It's going to show the available packages of the highest version we can current see. That's fine. There is just an issue with 2.3.3.5, and what appears to be only 2.3.3.5 that when you go to install the optional packages, it forces an update of the NCP package only to the latest version available. This shouldn't happen. I would definitely work with TAC to ensure the right packages are installed this time, even if we have to do it manually.