set devices global-settings out-of-sync-commit-behaviour accept??

[equinix]

Hi All,

I wanted to see what the larger community thinks about using this command? The reason I bring this up is that in our network we see a lot of manual commits to the devices our NSO manages. Obviously if a service config is manually modified that is not the same as what NSO has in the CDB, then we would see associated RPC errors.

When we first installed NSO, the intention was to have users not touch the devices to avoid out-of-sync conditions and in cases that they had to we would sync-from the device for configuration consistency. But the reality of enforcing this is challenging to say the least. We found out that this option was turned on because we kept noticing the sync status of many devices as "unknown".

Are there any reasons to remove this command and just accept out of sync manual commits? Are there any compelling reasons to either use this global setting or not?

[Jan Lindblad]

It's never easy to live in the real world. In the ideal world, the out-of-sync-commit-behavior shouldn't matter, because all changes would happen through the automation, and things wouldn't be out of sync to begin with.

Coming back to the real world, having the out of sync check on is a safety guard against NSO trying changes that have to be backed out moments later, possibly causing bad effects, or worse, that changes succeed but do the wrong thing. Turning off these safety guards increases the probability that something bad happens.

Some devices don't support this feature, and some even say they support it, but does it incorrectly, so for them it has to be off regardless. If you should have it on for devices that do support it is a business call. Do you want to prevent NSO touching devices when the computations are based on potentially bad input data. Or do you want NSO to try to push anyway, without guarantees. There are other safety checks that can be enabled (e.g. no-overwrite), and they might be provide a better balance for your business needs.

Different organizations run their business and networks differently. If you are automating a car factory, you might not want to go for 100% automation anytime soon, so would mix manual and automated steps. You could introduce robots in some areas of the factory, e.g. painting, but leave others manual, e.g. wheel mounting. The automation can work really well even in this mixed form as long as people are kept out of painting and robots kept out of wheel mounting.

[lmanor]

There is a good video from NSO Developer Days 2018 that discusses the topic of 'out of band' changes (not thru NSO) causing out-of-sync and available mechanism to deal with them: 

https://www.youtube.com/watch?v=ExdryBzvRRo

[joepak]

Hi,

The out-of-sync-commit-behaviour when set to accept will have the device report back as unknown after a check-sync is issued due to the fact that the transaction ID is no longer being tracked by NSO. To solve this issue, you can perform one of the following actions.


  1.  For the device, set the out-of-sync-commit-behaviour to reject. However, if out of band configuration changes (not initialized from NSO) are made, we will then get an out-of-sync message if device changes are saved to the device.

  2.  If the device has the out-of-sync-commit behavior set to accept, perform a device compare config (to determine what configuration the device has that NSO does not have), followed by a sync-to or a sync-from action (depending on what configuration you want saved to the device).

      leaf out-of-sync-commit-behaviour {
        type enumeration {
          enum reject;
          enum accept;
        }
        description
          "Specifies the behaviour of a commit operation involving a
           device that is out of sync with NCS. Value accept assumes that
           the device's sync state is unknown and it is cleared on commit.
           The default behaviour is to reject such commits.";
      }

I would assume the benefit would be to still be able to apply changes to the NSO CDB and then perform sync-from/to to fix the issue. Rather than waiting to have to fix an out-of-sync device, you can simply apply changes first (into CDB) and then connect to the device again.

[rogaglia]

From my point of view, the default behavior for NSO is set on a assumed operational environment where the tool will be working on. Your comment clearly state that these assumptions are not true for your case.

So, you should use this command but not alone: you should define a new set of operational practices. These practices may involve operational processes or NSO scheduling tasks to continue to make your overall operation safe. NSO has added new features that would make your life easier: partial-sync-from, commit no-overwrite, commit-queues and others. However, it is hard to recommend the right mix for your environment.

I hope this helps!

Roque