cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements

2314
Views
20
Helpful
4
Replies
Highlighted
Beginner

CSCvh98912 - Git734 DNAC upgrade fails w/ sftp-service not found and pod updates may not add or remove containers

Hi guys,

 

I struggled several days with upgrade of the DNA appliance from 1.1 to 1.1.2, but in the end I found a working workaround for this bug.

 

From the CLI I could see the system was failing upgrading:

 

$ maglev package status

maglev-1 [main - https://kong-frontend.maglev-system.svc.cluster.local:443]

NAME DEPLOYED AVAILABLE STATUS
-----------------------------------------------------------------------------------
application-policy - 2.1.4.170015 NOT_DEPLOYED
assurance 1.0.5.503 1.0.5.630 DEPLOYED
automation-core 2.1.1.60067 2.1.4.60025 DEPLOYED
base-provision-core - 2.1.4.60025 NOT_DEPLOYED
command-runner - 2.1.4.60025 NOT_DEPLOYED
device-onboarding - 2.1.4.60025 NOT_DEPLOYED
image-management 2.1.1.60067 2.1.4.60025 DEPLOYED
ncp-system 2.1.1.60067 2.1.4.60025 DEPLOYED
ndp-base-analytics 1.0.6.342 1.0.7.871 DEPLOYED
ndp-platform 1.0.6.246 1.0.7.803 DEPLOYED
ndp-ui 1.0.6.454 1.0.7.949 DEPLOYED
network-visibility 2.1.1.60067 2.1.4.60025 DEPLOYED
path-trace 2.1.1.60067 2.1.4.60025 DEPLOYED
sd-access - 2.1.4.60025 NOT_DEPLOYED
sensor-assurance - 1.0.5.326 NOT_DEPLOYED
sensor-automation - 2.1.4.60025 NOT_DEPLOYED
system 1.0.4.776 - UPGRADE_ERROR - maglev_workflow.workflow.exceptions.TaskCallableExecutionError: (1520330534.0461926, 1520330624.4249065, 'TimeoutError', 'Timeout of 90 seconds has expired while watching for k8s changes for telemetry-agent ')

 

I looked which parts were failing:

 

$ magctl appstack status | grep -v Run
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE
maglev-system agent-6kz9k 0/1 ImagePullBackOff 0 1m <none> 10.22.30.10
maglev-system remedycontroller-419825855-cb5sn 0/1 ImagePullBackOff 0 9m 10.22.27.69 10.22.30.10
maglev-system workflow-ui-2875804863-dxn68 0/1 ImagePullBackOff 0 9m 10.22.27.120 10.22.30.10

 

 I decided to try to pull the failing packages manually one by one.

 

$ maglev catalog system_update_addon pull image-remedy-controller-1-0-4-776
SystemUpdateAddon pull initiated

 

When all packages were in the Running state I upgraded the system again:

 

$ maglev package upgrade system
Package will start getting upgraded momentarily

 

I had to repeat those steps several times but in the end I was up and running:

 

$ maglev package status

maglev-1 [main - https://kong-frontend.maglev-system.svc.cluster.local:443]

NAME DEPLOYED AVAILABLE STATUS
-----------------------------------------------------------------------------------
application-policy - 2.1.4.170015 NOT_DEPLOYED
assurance 1.0.5.503 1.0.5.630 DEPLOYED
automation-core 2.1.1.60067 2.1.4.60025 DEPLOYED
base-provision-core - 2.1.4.60025 NOT_DEPLOYED
command-runner - 2.1.4.60025 NOT_DEPLOYED
device-onboarding - 2.1.4.60025 NOT_DEPLOYED
image-management 2.1.1.60067 2.1.4.60025 DEPLOYED
ncp-system 2.1.1.60067 2.1.4.60025 DEPLOYED
ndp-base-analytics 1.0.6.342 1.0.7.871 DEPLOYED
ndp-platform 1.0.6.246 1.0.7.803 DEPLOYED
ndp-ui 1.0.6.454 1.0.7.949 DEPLOYED
network-visibility 2.1.1.60067 2.1.4.60025 DEPLOYED
path-trace 2.1.1.60067 2.1.4.60025 DEPLOYED
sd-access - 2.1.4.60025 NOT_DEPLOYED
sensor-assurance - 1.0.5.326 NOT_DEPLOYED
sensor-automation - 2.1.4.60025 NOT_DEPLOYED
system 1.0.4.776 - DEPLOYED

 

I hope this guide will keep you from having to reimage the server from the ISO.

 

Regards

Michal

4 REPLIES 4
Highlighted
Beginner

Thanks mifi, it really works.

 

Regards

Martin

Highlighted
Beginner

Hi All,

 

We are trying to install DNAC 1.1.8, we are facing issue in deploying automation-core version 2.1.12.60011. Automation-core throws an error stating "DEPLOYMENT_ERROR - maglev.workflow.workflow.exceptions.TaskCallableExecutionError: (1547186922.4240553, 1547191113.6825242, 'TimeoutError' , 'Maximum wait time 4170 exceeded for the following services to be ready: task-service, telemetry-service, distributed -cache-service, file-service')". We even tried to pull each package from the catalog which failed by giving "maglev catalog system_update_addon pull image telemetry-service:2.1.12.60011", that too throws error saying "An unexpected error occurred".

I have attached the screenshots for your reference, it would be great if anyone can give us some insight about the issue.

 

Regards

Akchaiah

Highlighted

Hi,

 

Did you ever solve the issue with the automation-core package not deploying correctly as I have the same issue?

 

Thanks

Highlighted

No we were not able to solve that issue
Content for Community-Ad
Cisco Community October 2020 Spotlight Award Winners