Perform sync-from before python based service instantiation

[Ben Kelly]

I'm trying to perform a sync from on a device before the instantiation of a python based service. I'm using the cb_pre_modification to do it with this code:

@service.pre_modification

    def cb_pre_modification(self, tctx, op, kp, root, proplist):

        self.log.info('Service premod(service=', kp, ')')

        self.log.info('Checking device {} is in sync'.format('svlngen4-fab6-dmzdc-02-fw1'))

        with ncs.maapi.single_write_trans('admin', 'python') as trans:

            root = ncs.maagic.get_root(trans)

            device = root.devices.device['svlngen4-fab6-dmzdc-02-fw1']

            # check_sync_output = device.check_sync.request()

            sync_output = device.sync_from.request()

            self.log.info('Device has now been sync\'d: {}'.format(sync_output.result))

However, the config never gets pushed to the device. I believe that we're getting some sort of timeout, as there's 2 minutes from the time the sync-from is started until it ends:

2017-09-28 07:28:19 - fw_acl_03 - INFO - Checking device svlngen4-fab6-dmzdc-02-fw1 is in sync

2017-09-28 07:30:21 - fw_acl_03 - INFO - Device has now been sync'd: True

In the meantime the transaction times out and not configuration is pushed. The next time I try to commit something on that service, I get:

no registration found for callpoint fw_acl_03-servicepoint/service_create of type=external

I have to reload the packages to clear this error.

Is this the right approach to performing a sync-from before committing config changes to a device?

[alam.bilal · Accepted solution]

Hi There,

The above is due to a classic deadlock situation.

When a transaction is started, various locks are taken. At some stage the preMod callback runs. It tries to do a sync-from which in turn tries to grab the same locks.

In short, we cannot do updates to CDB (sync-from) while a transaction is in progress.

I have been in a similar situation before and had to find an alternative way to think about the problem..

Why is the sync-from needed while we are withing the service update?

A messy way (which I would not recommend) is to write action wrappers. The wrapper will first do a sync-from and then invoke the service CRUD.

Thanks.

[Ben Kelly]

The device that I'm pushing the configuration to is only partially managed by NSO. There are other automated systems as well as manual configuration adjustments going on. The service I'm working on makes adjustments to a shared ACL and at very specific line numbers. It's very likely that the ACL has been manually changed since the last time NSO committed a change and therefore the line numbers on the device won't match what's in the CDB.

So the idea is that before making any changes, NSO should first get an updated config from the device.

[alam.bilal]

I have worked on a use-case with slightly similar requirements. However there was clear demarcation of the configuration space touched between NSO and other orchestrator/manual-operator. Plus from a process perspective, particular naming conventions were to be used to avoid cross scribbling.

Some suggestions on the options to consider:

1. From the NB-System, first do a sync-from on the device[s] of interest. Then perform the service CRUD.

2. If don't want an extra API call from the NB System to do sync-from then within the service instance's CRUD, in the pre-mod callback a service "deep-check-sync" can be done to detect any scribbling. If scribbling detected then terminate the transaction and return delta-diffs.There will be performance implication associated with reading the device[s] configs.

3. To avoid performance penalty of (2), within the service-CRUD, in the pre-mod ensure all devices of interest are in-sync then perform service check-sync against the CDB.  Reject request if a device is out-of-sync or service not in sync. The NB-System will have to re-attempt the service-CRUD request. Also an optimisation is to have kicker watching the service instance node and performs sync-from post the service-CRUD. So the next attempt by NB-System can proceed.

4. Craft and action that first does the sync-from and them performs the service-CRUD. Downside, a lot of action wrapper code.

[randreetta]

easy and straightforward