Re: NSO - JSONRPC new write transaction with lock

LoicChabert27318 · ‎11-18-2020

Hi Nso dev,

I'm using json-rpc with react and read only mode is fine. Now, i try to create some objects/services on NSO 4.6 backend... In order to manage user commit concurrency, i want to lock the whole configuration and reject new write transaction if someone else is editing config tree.

Following the documentation, 'new_trans' json-rpc method allow exclusive flag. An error should be raised if configuration is lock by someone else:

{"type": "db.locked", "data": {"sessions": <array of string>}}

I have tried to reproduce the case with the following python script:

import requests
import random
import time

s = requests.session()

print("Login into NSO WebUi")
login_data = {"jsonrpc":"2.0","method":"login","id":random.randint(200, 65544),"params":{"user":"user","passwd":"password"}}
aaa = s.post("http://10.204.10.21:8808/jsonrpc", json=login_data)

for i in range(1,20):
   print("Open Write transaction %d" % i)
   data = {"jsonrpc":"2.0","method":"new_trans","id":random.randint(200, 65544),"params":{"db":"candidate","mode":"read_write","conf_mode":"exclusive","on_pending_changes":"reject"}}
   print(data)
   r = s.post("http://10.204.10.21:8808/jsonrpc", json=data)
   print(r.json())
   th = r.json()["result"]["th"]
   time.sleep(10)

print("Logout into NSO WebUI")
data = {"jsonrpc":"2.0","method":"logout","id":random.randint(200, 65544)}
s.post("http://10.204.10.21:8808/jsonrpc", json=data)

And results:

python3 lock.py
Login into NSO WebUi

Open Write transaction 1
{'params': {'conf_mode': 'exclusive', 'on_pending_changes': 'reject', 'mode': 'read_write', 'db': 'candidate'}, 'id': 58763, 'jsonrpc': '2.0', 'method': 'new_trans'}
{'result': {'th': 1}, 'id': 58763, 'jsonrpc': '2.0'}

Open Write transaction 2
{'params': {'conf_mode': 'exclusive', 'on_pending_changes': 'reject', 'mode': 'read_write', 'db': 'candidate'}, 'id': 59148, 'jsonrpc': '2.0', 'method': 'new_trans'}
{'result': {'th': 2}, 'id': 59148, 'jsonrpc': '2.0'}

Open Write transaction 3
{'params': {'conf_mode': 'exclusive', 'on_pending_changes': 'reject', 'mode': 'read_write', 'db': 'candidate'}, 'id': 59485, 'jsonrpc': '2.0', 'method': 'new_trans'}
{'result': {'th': 3}, 'id': 59485, 'jsonrpc': '2.0'}

But, over cli, behaviour is normal:

lchabert@ncs> configure exclusive
Error: configuration database locked by:
user http (webui from 127.0.0.1) on since 2020-11-06 17:35:50
Aborted: configuration locked
[error][2020-11-06 18:08:54]
lchabert@ncs>

Is it a normal behaviour with json-rpc interface ?

Why the json-rpc lock error is never raised on exclusive mode ?

Any mis-understanding of the documentation ?

Thanks in advance,

Best regards.

perander · ‎11-23-2020

There is currently no way to achieve the same behaviour in JSON-RPC as the CLI has,
i.e. aborting a new transaction in exclusive conf mode when an exclusive lock is taken.

The transaction will however fail "commit" and "validate_commit", the error message
for "commit" isn't particularly helpful but the error message for "validate_commit" is
basically the same as the CLI error message.

Furthermore, it is possible to call "resolve_trans" when the exclusive lock is released
and then commit will work.

It can be discussed whether the behaviour should be this way or not, or if one should be
able to choose. Please connect with TAC for further discussion.

Perhaps commit queues could be an alternative?

LoicChabert27318 · ‎11-24-2020

Thanks for your answer, i will take a look on validate_commit message when exclusive lock is taken.

I'm using commit queues to release webinterface and display user notification on task background. But i have two questions concerning commit queues:

Commit tasks queued are executed with FIFO queue style ?
How config tree conflits are managed ? If two commits are pushed in the same time with incompatible configuration, which one will fail, and which one will succed ? If one failed, is there any methods for end users to not loose their changes ?

Thanks in advance,

Regards.

perander · ‎11-24-2020

It is also possible to explicitly take a lock with the lock_db method. It will fail if it's not
possible to take the lock.

Furthermore the documentation states the following when using on_pending_changes=reject:

The on_pending_changes param decides what to do if the candidate already
has been written to, e.g. a CLI user has started a shared configuration session
and changed a value in the configuration (without committing it). If this parameter
is omitted the default behavior is to silently reuse the candidate. If "reject" is
specified the call to the "new_trans" method will fail if the candidate is non-
empty. If "discard" is specified the candidate is silently cleared if it is non-empty.

Regarding commit queues see the youtube video for an introduction.

https://www.youtube.com/watch?v=9pEdvZS5yE0

perander · ‎11-25-2020

Note however that the candidate datastore is not available in NSO.

I realize that the webui docs need to be changed to reflect this and
have filed a ticket to fix them.

How did you manage to setup a new transaction with candidate as db?

LoicChabert27318 · ‎11-25-2020

I don't have any special magic trick, just posting the candidate JSON as shown on my first post

But no error raised from nso. Maybe NSO should raised an error ?

Thanks for the video. I will watch it as soon as possible.

If the speaker of video pass here, I can't find any source code. Could you please release it ? @dansullivan