Persistence of ncs:post-action-node in Nano services
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-26-2024 06:30 AM
Hi,
I'm trying to get more experience with Nano services and have run into a problem with persisting data from
ncs:state "my-nano:cpe-created" {
ncs:create {
ncs:nano-callback;
ncs:pre-condition {
ncs:all {
ncs:monitor "$SERVICE" {
ncs:trigger-expr "cpe != ''";
}
ncs:monitor "$SERVICE" {
ncs:trigger-expr "deployment-ip != ''";
}
}
}
ncs:post-action-node "/ncs:devices/ncs:device[ncs:name=$CPE]/ssh" {
ncs:action-name "fetch-host-keys";
ncs:result-expr "not(result = 'failed')";
ncs:sync;
}
}
}
Later I then have a state that does a sync-from
ncs:state "my-nano:device-synced" {
ncs:create {
ncs:post-action-node "/ncs:devices/device[name=$CPE]" {
ncs:action-name "sync-from";
ncs:result-expr "result = 'true'";
ncs:sync;
}
ncs:pre-condition {
ncs:monitor
"$SERVICE/plan/component[type='my-nano:cpe']/state[name='my-nano:cpe-created']" {
ncs:trigger-expr "status = 'reached'";
}
}
}
}
I can see in the logs that "fetch-host-keys" is run and evaluated my plan reaches "create-reached"
cpe cpe false - init reached 2024-07-26T13:18:24 - -
cpe-created reached 2024-07-26T13:18:24 - create-reached
device-synced reached 2024-07-26T13:18:25 - failed
ready not-reached - - -
plan failed
plan error-info message "Error running action for side-effect item /ncs:services/my-nano:my-nano{1231231231}/plan/component{my-nano:cpe cpe}/state{my-nano:device-synced}: result:false info:<<\"Failed to authenticate towards device bingo: Unknown SSH host key\">> "
plan error-info log-entry /services/my-nano:my-nano[ssid='1231231231']/log/log-entry[when='2024-07-26T13:18:26.254904+00:00']
My xpath.trace says:
26-Jul-2024::15:18:25.004 get_elem(/devices/device[name='bingo']/ssh/fetch-host-keys/$output/result) = updated
But my sync-from fails because ssh keys are never persisted to cdb..
What am I doing wrong? Been banging my head against this issue a couple of days now
Behaviour is the same with NSO 6.3 as it is with 6.1
If run the demo from examples.ncs/development-guide/nano-services/mpls-vpn-vrouter it does seem like it has the same behavior.. SSH keys and result of sync-from is not persisted to CDB.
admin@ncs> show devices list
NAME ADDRESS DESCRIPTION NED ID ADMIN STATE
--------------------------------------------------------------
ce0 127.0.0.1 - cisco-ios-cli-3.8 unlocked
ce1 127.0.0.1 - cisco-ios-cli-3.8 unlocked
ce2 127.0.0.1 - cisco-ios-cli-3.8 unlocked
ce3 127.0.0.1 - cisco-ios-cli-3.8 unlocked
esc0 127.0.0.1 - esc-nc-1.0 unlocked
p0 127.0.0.1 - cisco-iosxr-cli-3.5 unlocked
pe0 127.0.0.1 - cisco-iosxr-cli-3.5 unlocked
[ok][2024-07-26 15:22:07]
admin@ncs> request devices sync-from
sync-result {
device ce0
result true
}
sync-result {
device ce1
result true
}
sync-result {
device ce2
result true
}
sync-result {
device ce3
result true
}
sync-result {
device esc0
result true
}
sync-result {
device p0
result true
}
sync-result {
device pe0
result true
}
[ok][2024-07-26 15:22:18]
admin@ncs> configure
Entering configuration mode private
[ok][2024-07-26 15:22:21]
[edit]
admin@ncs% load merge vpn_volvo.xml
[ok][2024-07-26 15:22:25]
[edit]
admin@ncs% commit
Commit complete.
[ok][2024-07-26 15:22:26]
[edit]
admin@ncs%
System message at 2024-07-26 15:22:26...
Commit performed by admin via system using cli.
admin@ncs%
System message at 2024-07-26 15:22:27...
Commit performed by admin via console using cli.
admin@ncs%
System message at 2024-07-26 15:22:29...
Commit performed by admin via console using cli.
admin@ncs%
System message at 2024-07-26 15:22:29...
Commit performed by admin via system using cli.
admin@ncs%
System message at 2024-07-26 15:22:30...
Commit performed by admin via console using cli.
admin@ncs%
System message at 2024-07-26 15:22:30...
Commit performed by admin via system using cli.
admin@ncs%
System message at 2024-07-26 15:22:30...
Commit performed by admin via system using cli.
admin@ncs%
System message at 2024-07-26 15:22:30...
Commit performed by admin via system using cli.
admin@ncs%
System message at 2024-07-26 15:22:30...
Commit performed by admin via console using cli.
admin@ncs%
System message at 2024-07-26 15:22:31...
Commit performed by admin via system using cli.
admin@ncs%
System message at 2024-07-26 15:22:31...
Commit performed by admin via system using cli.
admin@ncs%
System message at 2024-07-26 15:22:31...
Commit performed by admin via system using cli.
admin@ncs%
System message at 2024-07-26 15:22:31...
Commit performed by admin via console using cli.
admin@ncs%
System message at 2024-07-26 15:22:32...
Commit performed by admin via console using cli.
admin@ncs% exit
[ok][2024-07-26 15:23:43]
admin@ncs> show devices list
NAME ADDRESS DESCRIPTION NED ID ADMIN STATE
----------------------------------------------------------------------------
ce0 127.0.0.1 - cisco-ios-cli-3.8 unlocked
ce1 127.0.0.1 - cisco-ios-cli-3.8 unlocked
ce2 127.0.0.1 - cisco-ios-cli-3.8 unlocked
ce3 127.0.0.1 - cisco-ios-cli-3.8 unlocked
esc0 127.0.0.1 - esc-nc-1.0 unlocked
p0 127.0.0.1 - cisco-iosxr-cli-3.5 unlocked
pe0 127.0.0.1 - cisco-iosxr-cli-3.5 unlocked
volvo_vpn_CSR_esc0 127.0.0.1 - cisco-ios-cli-3.8 unlocked
[ok][2024-07-26 15:23:52]
admin@ncs> show configuration devices device volvo_vpn_CSR_esc0 ssh
host-key-verification none;
[ok][2024-07-26 15:23:53]
admin@ncs> request devices device volvo_vpn_CSR_esc0 ssh fetch-host-keys
result updated
fingerprint {
algorithm ssh-ed25519
value 72:ae:81:76:58:2e:5c:e2:fd:68:a0:a8:6c:f1:fe:fe
}
[ok][2024-07-26 15:24:01]
admin@ncs> request devices device volvo_vpn_CSR_esc0 sync-from dry-run
cli config {
ip {
+ source-route true;
}
}
[ok][2024-07-26 15:24:12]
- Labels:
-
Crosswork NSO
-
Other NSO Topics
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-10-2024 07:38 AM
Dug a bit deeper into this, and I'm still puzzled..
In examples.ncs/development-guide/nano-services/mpls-vpn-vrouter/packages/vm-manager there is src/java/src/com/cisco/vmmanager/escstart.java
The only reason this example works is because in line 382 host-key-verification is disabled:
vm.container("ssh").leaf("host-key-verification").sharedSet("none")If I comment out that line, the example will fail as my own service
admin@ncs> show vm-manager start plan | tab
BACK POST ACTION
NAME FAILED MESSAGE LOG ENTRY TYPE NAME TRACK GOAL STATE STATUS WHEN ref STATUS ID CREATED FROM LOG ENTRY MESSAGE
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
volvo_vpn_CSR X Error running action for side-effect item /vmm:vm-manager/start{volvo_vpn_CSR}/plan/component{ncs:self self}/state{vmm:device-keys}: result:false info:<<"Failed to authenticate towards device volvo_vpn_CSR_esc0: Unknown SSH host key">> /vm-manager/start[name='volvo_vpn_CSR']/log/log-entry[when='2024-09-10T14:37:57.57108+00:00'] self self false - init reached 2024-09-10T14:37:52 - -
init-vm reached 2024-09-10T14:37:52 - -
vm-initialized reached 2024-09-10T14:37:56 - -
device-created reached 2024-09-10T14:37:57 - -
device-ready reached 2024-09-10T14:37:57 - create-reached
device-keys reached 2024-09-10T14:37:57 - failed
device-synced not-reached - - -
ready not-reached - - - 2024-09-10T14:37:57.571333+00:00 /side-effect-queue/side-effect[id='7'] /vm-manager/start[name='volvo_vpn_CSR']/log/log-entry[when='2024-09-10T14:37:57.57108+00:00'] Error running action for side-effect item /vmm:vm-manager/start{volvo_vpn_CSR}/plan/component{ncs:self self}/state{vmm:device-keys}: result:false info:<<"Failed to authenticate towards device volvo_vpn_CSR_esc0: Unknown SSH host key">>
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-10-2024 07:41 AM
I've tried adding "ncs:commit" to my ncs:state create - makes no difference.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-10-2024 07:59 AM
I believe your problem comes from your pre-condition setting:
ncs:pre-condition {
ncs:monitor
"$SERVICE/plan/component[type='my-nano:cpe']/state[name='my-nano:cpe-created']" {
ncs:trigger-expr "status = 'reached'";
}
}The plan file status is updated before the post-action (fetch ssh key) is finished, so you have a raise condition.
As shown in the example, you should monitor that the post-action finished:
ncs:trigger-expr "post-action-status = 'create-reached'";
So, I would modify the monitor section of your sync-from state to actually monitor that the post-action is done.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-10-2024 01:57 PM
Thanks for your reply. Good point.
I've updated my monitor to look for "post-action-status" instead. Makes no difference
I've tried altering my plan to only create the device, fetch host keys, and then enter ready state. All nano steps are processed but still no ssh keys in the configuration.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-10-2024 04:10 PM
Hi, one possibility is that the ssh key is failing just because there is a key algorithm mismatch. This is a common problem due to recent security tightening rules both in NSO and the devices. So, please check the ned traces an test that it works from the device even outside of the nano.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-11-2024 06:50 AM
The device is a netsim device. I can fetch ssh keys and sync-from manually without any issues..
