call partial_sync_from function in cb_create

[dongwi]

Hi Team:

if i call the partial_sync_from in the cb_create func, the paragram will stop at the executing.

the code is :

partial_func = root.ncs__devices.partial_sync_from
input = partial_func.get_input()
input.path = ["/ncs:devices/ncs:device[ncs:name='%s']/config/vrp:route-policy/" % service.device]
self.log.info('before exec the partial_sync_from func')
result = partial_func(input)
for r in result.sync_result:
    if not r.result:
        self.log.error('%s partial sync from device failed' % service.device)

the vm log:

ncs-dp-26133-nms:router in-1-th-3578: - before exec the partial_sync_from func

and cannot be interrupted by ctrl + c

and my question is: Is there any method i can call the actions in cb_create ?

thanks!

[lmanor]

You are in essentially creating a lock deadlock when you attempt to execute an action within the create_cb:

You start execution of a create callback... this starts a transaction which acquires the global lock...

    While executing within this global lock...

        You call issue a call to an action... which, you guessed it, starts another transaction

              This new transaction will attempt to acquire the global lock - and since it is already in-use by your create callback... it will wait for it to become available.

               But, the create callback will never complete because this action is waiting... forever for the lock.

Calling a function or action (if it could be) that takes a large time (in excess of milliseconds) is highly frowned upon in NSO create code and would obviously be devastating to NSO transaction throughput performance.  The create_cb() call should be streamline and get out of the critical section (global lock) as quickly as possible.

To handle time expensive operations such as sync-from (which by the way is generally not good practice to do on each create call - again for performance and risk issues (what are you getting on sync-from?)) the action must be called from an action or handled via reactive-fastmap staging.

[dongwi]

thank you Imanor.

From your reply, I get some point
1. service can create another service (equals call another service's create_cb func) in the create_cb() func, but cannot call the action.
2. action can call another action, but cannot create service
Am i right?

In my case, i have an input param which comes from the device, and i want to ensure the param is newest, so i call the sync-from action. may the RFM is the only way to get the purpose?

[vleijon]

No, in general you can call actions from create code (though it is often not recommended) and write services from actions.

The important thing, just as in ghostbusters, is to not cross the streams - you should make sure that your action does not start another transaction while the database is locked.

In this case, just to make doubly sure, in your initial example, where did you get the root object? As a rule: Make sure you are not starting new transactions inside the create method (unless you really know what you are doing).

[dongwi]

Thx vleijon. I get the point: do not start new transactions inside a transactions.

In this case, the pseudo code is :

@service.create
def cb_create(self, tctx, root, service, proplist):
    partial_func = root.ncs__devices.partial_sync_from
    input = partial_func.get_input()
    input.path = ["/ncs:devices/ncs:device[ncs:name='%s']/config/vrp:route-policy/" % service.device]
    self.log.info('before exec the partial_sync_from func')
    result = partial_func(input)
    for r in result.sync_result:
        if not r.result:
            self.log.error('%s partial sync from device failed' % service.device)

The create code runs within the global transaction lock, then partial_sync_from func start new transaction which wants acquire the global transaction lock too. deadlock, just like Imanor said.

Then I have an another question:
If the global transaction lock is a ReentrantLock, the partial_sync_from func can acquire the global transaction lock?

Ps: Is there any document illustrate which action included in nso will start a new transaction? Or the actions which may modify the database imply it will start new transactions.

[vleijon]

The lock belongs to the transaction, and it isn't exactly reentrant but an action (or any other code) can choose to attach to an existing transaction (using the Maapi.attach method in python). 

It would be nice if the yang file mentioned how an action behaves in the description of the action, in the case of partial-sync-from it is documented in nso_development-5.1.0.1.pdf (a section called Partial sync-from) and says among other things "Pulling the configuration from the network needs to be initiated outside the service code," but does not elaborate on the motivation behind it (and i don't know myself).

One potential problem is that if you attach to a transaction owned by a service the changes from the action will be part of the diff-set of that service, and in the case of the partial-sync-from you probably don't want that to happen.

[dongwi]

Thanks for your reply again, vleijon.

1. If each transaction have to acquire a global lock? If so, is there other reason except thread-safe?
2. From the Action.action wrapper func, I found that there have to be a transaction contained in the action object. Am i right?
2. In the case, the partial-sync-from func is got from the root object which contains a transaction attribute, so, in my opinion, the func will inherit from the root object normally. But It seems that the action will start a new trans.

Would you show me the code how to use the Maapi.attach method in python? Because in the create_cb func, there is not Maapi object anymore.

[vleijon]

The lock is actually an implementation strategy for a key property of NSO: A brief summary is something like this: Each transaction is guaranteed global consistency of the data it writes, both to the network and to the database in general. This greatly simplifies the programming model for the developer of create code. It means that we can share important information between services without the need for the developer to care about synchronisation.  We can also write validation hooks and expression that work on the entire database without having to worry about having a partial or mixed view of the data.

Each action has to be called from a session so there has to be something, but there is no guarantee that it is a read/write transaction and no guarantee that it is against the configuration datastore. 

There is definitely no automatic inheritance or reuse of transactions for actions and there are sometimes complicated technical reasons why you want a fresh transaction.

This is the start of an action that I once wrote that attaches to the underlying transaction:

@Action.action
def cb_action(self, uinfo, name, kp, input, output):
m = maapi.Maapi()
m.start_user_session(uinfo.username, 'test_context')
t = m.attach(uinfo.actx_thandle, 0, uinfo.usid)