06-27-2017 04:04 PM - edited 03-01-2019 03:53 AM
A common query from NSO developers and users is about rollback, as different systems handle it differently.
E.g. If a change is made, then a rollback is needed, does NSO perform the equivalent “no” commands to remove the config, or does a pre-change snapshot of the config “merge with” or “replace” the new config with the initial change implemented.
Solved! Go to Solution.
06-27-2017 05:17 PM
When answering this question, I think it’s helpful to understand there are a few different concepts at work in NSO. Unfortunately “rollback” gets used generically to mean a whole number of different things.
Fundamental: Transactions
NSO transactions are distributed (or multi-phase). An analogy would be in arranging a meeting. Instead of just saying straight away “come to this meeting” (and then you’re the only one that shows up - let's try again!), it’s more like asking everyone “can you make this meeting?” (prepare) and only once everyone responds “yes”, does NSO say “come to this meeting” (apply).
CDB Rollback
To keep this example simple, let’s pick something that doesn’t result in a change anywhere outside of the CDB: a NED setting.
When we request a change to a NED setting, via the CLI, a transaction is created containing the change that we eventually commit. Your CLI session now has something like a "candidate" view of the CDB - as if your change had already been applied. When we commit, the CDB validates that it is an acceptable change set (“prepare”s it). If it’s not valid, you get an error (e.g. you might have specified an invalid value), and the change is not applied – the “running” CDB was never touched (just your "candidate" view was). If it’s valid, then it gets applied to the running CDB. During that apply, the reverse action is calculated and stored in a rollback file on the NSO filesystem. If at some time later, you want to revert the change, you can go and identify the commit, and request the selective rollback of that commit, or rollback that commit and ALL changes since that commit. By default, NSO stores 500 of these rollback files. (See: NSO User Guide - Modifying the configuration - ~pg11 for details of viewing and applying rollbacks).
The CDB rollback should be used carefully by an end user, because subsequent transactions since the rollback was saved may have changed values that are not meant to be rolled back. Hence the existence of the Service Manager. However, rollbacks are still important to revert a change to a service.
NED “Rollback” (a.k.a. revert changes in a device)
Let’s make this example a bit more complex. Let’s make three changes to the CDB, all changes to the devices tree for three separate devices. The type of change is not really relevant. The changes are made via the NSO CLI, a transaction is created containing the changes – as before. When you commit, the device manager creates a nested transaction for each device being changed within the device tree. Your CDB transaction includes these sub transactions - and each one now needs to be successfully “prepared”. Whilst the device manager is coordinating these nested transactions, it falls to the NEDs to actually carry out these transaction phases, and we have a few different types of NED.
In this example, Device 1 is a netconf device. Device 2 is a CLI NED for an OS that supports single phase transactions, and 3 is a non-transactional CLI NED (for more information, see the NED Developer Guide, which outlines the below mechanics in more detail).
During prepare:
However, on Device 3, something fails in one of the final commands - what happens now?
Special case: commit dry-run
A commit dry-run issues the prepare request... as we can see, in the prepare phase on CLI devices, we would normally push the commands – that’s not exactly a dry run!
The prepare request, issued to the Device Manager, for a dry-run is special, and the Device Manager prepares the expected changes exactly as it normally would, but doesn't "prepare" via the NED. The NED framework issues a special prepareDry command to the NED, containing the expected CLI commands, and it is up to the NED code to determine if any manipulation needs to be done to the CLI generated by the Device Manager, to produce accurate dry-run results.
Special case: commit commit-queue
When the commit queue is used, things get a bit tricky with reverting device level issues.
In NSO pre-4.4:
The challenge is, that in NSO pre-4.4, as described above, the Device Manager/NED needs access to the the NSO view of the device before the commit (as-was), to calculate the reverse diff (as in Device 3 above). When a commit queue is used, the current "running" view in the CDB will not represent that - the "candidate" view (the to-be config) - was already written, so now it represents the to-be view, possibly with other changes applied.
If a commit queue was used in the above example, the commands to Device 3 would not correctly be reverted - potentially requiring a manual intervention to revert the changes to the device.
In NSO 4.4 and later:
A new mechanism has been added that means this is no longer such a special case. Please see the NSO CHANGE log for 4.4, and search for snapshot for full details. Essentially the CDB keeps a snapshot view of the device before each transaction entry in the queue (as-was), and uses that as-is view to calculate the rollback.
Services “Rollback”
Thirdly, we have Services. Services manage their own “rollback”s, via Fastmap. No rollback files are used in the below (unless you decided to invoke a rollback of a commit that added a service, which would infact issue a service delete).
It's helpful if we were all a bit more accurate when discussing “rollback” in the context of services, because there are a few discrete pieces:
Reactive “Rollback”
Finally, this all gets a bit more complex with Reactive Fastmap, which might need to put in place its own delete “workflow” to ensure the ordering of the “delete” issued to the underlying pieces are correct. This could be via CDB subscriber/kicker, or some sort of Nano Service.
06-27-2017 05:17 PM
When answering this question, I think it’s helpful to understand there are a few different concepts at work in NSO. Unfortunately “rollback” gets used generically to mean a whole number of different things.
Fundamental: Transactions
NSO transactions are distributed (or multi-phase). An analogy would be in arranging a meeting. Instead of just saying straight away “come to this meeting” (and then you’re the only one that shows up - let's try again!), it’s more like asking everyone “can you make this meeting?” (prepare) and only once everyone responds “yes”, does NSO say “come to this meeting” (apply).
CDB Rollback
To keep this example simple, let’s pick something that doesn’t result in a change anywhere outside of the CDB: a NED setting.
When we request a change to a NED setting, via the CLI, a transaction is created containing the change that we eventually commit. Your CLI session now has something like a "candidate" view of the CDB - as if your change had already been applied. When we commit, the CDB validates that it is an acceptable change set (“prepare”s it). If it’s not valid, you get an error (e.g. you might have specified an invalid value), and the change is not applied – the “running” CDB was never touched (just your "candidate" view was). If it’s valid, then it gets applied to the running CDB. During that apply, the reverse action is calculated and stored in a rollback file on the NSO filesystem. If at some time later, you want to revert the change, you can go and identify the commit, and request the selective rollback of that commit, or rollback that commit and ALL changes since that commit. By default, NSO stores 500 of these rollback files. (See: NSO User Guide - Modifying the configuration - ~pg11 for details of viewing and applying rollbacks).
The CDB rollback should be used carefully by an end user, because subsequent transactions since the rollback was saved may have changed values that are not meant to be rolled back. Hence the existence of the Service Manager. However, rollbacks are still important to revert a change to a service.
NED “Rollback” (a.k.a. revert changes in a device)
Let’s make this example a bit more complex. Let’s make three changes to the CDB, all changes to the devices tree for three separate devices. The type of change is not really relevant. The changes are made via the NSO CLI, a transaction is created containing the changes – as before. When you commit, the device manager creates a nested transaction for each device being changed within the device tree. Your CDB transaction includes these sub transactions - and each one now needs to be successfully “prepared”. Whilst the device manager is coordinating these nested transactions, it falls to the NEDs to actually carry out these transaction phases, and we have a few different types of NED.
In this example, Device 1 is a netconf device. Device 2 is a CLI NED for an OS that supports single phase transactions, and 3 is a non-transactional CLI NED (for more information, see the NED Developer Guide, which outlines the below mechanics in more detail).
During prepare:
However, on Device 3, something fails in one of the final commands - what happens now?
Special case: commit dry-run
A commit dry-run issues the prepare request... as we can see, in the prepare phase on CLI devices, we would normally push the commands – that’s not exactly a dry run!
The prepare request, issued to the Device Manager, for a dry-run is special, and the Device Manager prepares the expected changes exactly as it normally would, but doesn't "prepare" via the NED. The NED framework issues a special prepareDry command to the NED, containing the expected CLI commands, and it is up to the NED code to determine if any manipulation needs to be done to the CLI generated by the Device Manager, to produce accurate dry-run results.
Special case: commit commit-queue
When the commit queue is used, things get a bit tricky with reverting device level issues.
In NSO pre-4.4:
The challenge is, that in NSO pre-4.4, as described above, the Device Manager/NED needs access to the the NSO view of the device before the commit (as-was), to calculate the reverse diff (as in Device 3 above). When a commit queue is used, the current "running" view in the CDB will not represent that - the "candidate" view (the to-be config) - was already written, so now it represents the to-be view, possibly with other changes applied.
If a commit queue was used in the above example, the commands to Device 3 would not correctly be reverted - potentially requiring a manual intervention to revert the changes to the device.
In NSO 4.4 and later:
A new mechanism has been added that means this is no longer such a special case. Please see the NSO CHANGE log for 4.4, and search for snapshot for full details. Essentially the CDB keeps a snapshot view of the device before each transaction entry in the queue (as-was), and uses that as-is view to calculate the rollback.
Services “Rollback”
Thirdly, we have Services. Services manage their own “rollback”s, via Fastmap. No rollback files are used in the below (unless you decided to invoke a rollback of a commit that added a service, which would infact issue a service delete).
It's helpful if we were all a bit more accurate when discussing “rollback” in the context of services, because there are a few discrete pieces:
Reactive “Rollback”
Finally, this all gets a bit more complex with Reactive Fastmap, which might need to put in place its own delete “workflow” to ensure the ordering of the “delete” issued to the underlying pieces are correct. This could be via CDB subscriber/kicker, or some sort of Nano Service.
06-28-2017 01:45 AM
It is worth to notice that for a CLI NED some specific NED settings are possible to move the confirmation out of the prepare state into commit state. That is introduced for example with IOS-XR NED 6.0.1 where support for confirmed-in-commit method with the cisco-iosxr-commit-method ned-setting was added. What that does is following:
enum confirmed-in-commit { | |
tailf:info "Same as confirmed method except that 'commit confirmed' is" | |
+" called in NSO commit phase instead of prepare phase. Hence later" | |
+", which results in a shorter time until the commit is confirmed"; | |
} |
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide