Python API Maagic not updating module attributes after doing packages reload

[German Ruiz]

Hi all,

My team is working on a Web Server API to retrieve data about devices, services, modules from NSO. For this we are using Flask and the Python API (Maagic and Maapi modules). To start we are using a very simple module defined by this YANG:

module test {
  namespace "http://com/example/test";
  prefix test;

  list test {
    key name;

    leaf name {
      type string;
    }

    leaf number {
  	type uint32;
    }

    list interface {
  	key id;

  	leaf id {
  	    type string;
  	}

  	leaf value {
  	    type uint32;
  	}
     }
  }
}

This module gets loaded into NSO using the ncs-make-package but not as a service point, so with no template to map it to a service. So to get this module with Maagic we are opening a single read transaction the usual way and using:

test = ncs.maagic.get_root(t)['test']

All this works fine, we have a List object in test with our simple module instances.

Now the issue that we are having appears when we change the YANG model (By adding a new leaf or changing one) and then do the packages reload command. (First we do make -C packages/test/src and then do the packages reload, by the web ui or the cli).

Then if I go to the same endpoint to get data from this module in my Flask app I'm getting the module's old structure. We are opening a new transaction every time we hit an endpoint but Maagic still returns the test module without the changes. Something similar happens when using IPython. Without closing the session in IPython If we do the packages reload scenario, then we get a new transaction the usual way and try to parse the test module like this:

ncs.maagic.as_pyval(test) 

If the change in the YANG file is for example delete the number leaf, then we get an error like this:

Error: badly formatted or nonexistent path (8): Bad path element "number" after: /test

The object test show the number attribute but when trying to parse it fails as this does not exist in NSO anymore.

The workaround for us right now is to reset the Flask server (or IPython, so create a new process) every time we do a packages reload, which is not good as we want this WEB API always up so we can have the accurate data from NSO at any moment.

Things we try with no success so far:

1. Create new python thread every time a new transaction is needed

2. Re-importing ncs library using importlib every time a new transaction is needed.

3. Closing session and deleting maapi object every time

4. Using the clear cache private methods in maagic module.

Is it possible to detect a packages reload (upgrade) with maapi/maagic and if so, is there a way you can purge the data you have in your current session so that you get the updated one?

Thanks!

[vleijon]

There are some things you can call:

First of all there is a package called _ncs.events, and you can listen for an upgrade event.

There is a call in maapi called load_schemas() that might be able to update the schema information for you.

I haven’t really tried either of these for this purpose, so I can’t give much guidance.

[German Ruiz]

I just tried with load_schemas() and it does not seems to work for our case. From the function code it only does the load when 

_schemas_loaded is False. Checking the maapi object this variable is on True after doing:

m = maapi.Maapi()

So I changed it to True to force the load calling the load_schemas() function, but nothing happens (and _schemas_loaded stays False).

Other stuff I noticed doing some tests on IPython. I have an instance created from the simple module test with name test, so I can get this instance by doing this with a fresh transaction t:

test = maagic.get_root(t)['test']['test']

Then using dir(test) I can see the list of attributes:

In [125]: dir(test)                                                                                            
Out[125]: 
['__class__',
 '__delattr__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__get__',
 '__getattribute__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__le__',
 '__lt__',
 '__ne__',
 '__new__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__self__',
 '__self_class__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__thisclass__',
 'interface',
 'name',
 'number']

As you can see the 'number' leaf that I deleted from the YANG module is somehow still created as an attribute by maagic. Then if I try to get it with getattr:

In [128]: getattr(test, 'number')  
....
Error: badly formatted or nonexistent path (8): Bad path element "number" after: /test

So the same issue when using as_pyval(), because it uses getattr as well. The getattr in the Mappi object says that looks for the corresponding value in the low-level maapi module, so it could be that the high-level one is not getting updated against the low one?

Thanks!