NSO Service Upgrade with Python

[mleske]

Hello,

based on a discussion posted earlier in NSO Developer Hub ("Service Upgrade" with Python available?), I produced a full write-up on how to use the Python API to upgrade services, e.g. when YANG models change. This example is similar to the Java-based service upgrade show in .../examples.ncs/getting-started/14-upgrade-service.

The initial package and the upgraded packages including Python code are attached.

Thanks to Jan Lindblad for his guidance.

Mike

1. Create project and package

$ cd ~/dev/NSO/projects
$ ncs-setup --dest ./python-upgrade
$ cd python-upgrade/packages
$ ncs-make-package --service-skeleton python-and-template sample

2. Write Service YANG Module

  augment /ncs:services {
    list sample {
      description “This is an RFS skeleton service”;

      key name;
      leaf name {
        tailf:info “Unique service id”;
        tailf:cli-allow-range;
        type string;
      }

      uses ncs:service-data;
      ncs:servicepoint sample-servicepoint;

      // Define simple VRF parameters: Name and VRF-ID for route-target config
      leaf vrf-name {
        type string;
      }

      leaf vrf-id {
        type uint16 {
          tailf:info “<0-65535>;;VRF-ID”;
          range “0..65535”;
        }
      }
    }
  }

3. Print Service YANG Module as tree

$ pyang -f tree sample.yang
module: sample
augment /ncs:services:
   +--rw sample* [name]
      +--rw name                        string
      ...
      +--rw vrf-name?                   string
      +--rw vrf-id?                     uint16

4. Import service into NSO

$ ncs_cli -u admin

admin connected from 127.0.0.1 using console on MLESKE-M-Q005
admin@ncs> request packages reload

>>> System upgrade is starting.
>>> Sessions in configure mode must exit to operational mode.
>>> No configuration changes can be performed until upgrade has completed.
>>> System upgrade has completed successfully.
reload-result {
    package sample
    result true
}
[ok][2017-09-28 09:54:12]
admin@ncs>

5. Configure 2 Sample services

admin@ncs% set services sample 100 vrf-name vrf100 vrf-id 100
admin@ncs% set services sample 101 vrf-name vrf101 vrf-id 101
admin@ncs% commit

admin@ncs% show services sample
sample 100 {
    vrf-name vrf100;
    vrf-id   100;
}
sample 101 {
    vrf-name vrf101;
    vrf-id   101;
}

6. Duplicate “sample” package to prepare YANG + Python Upgrade

$ cp -R sample upgrade/sample-v1  # This will be our backup
$ cp -R sample upgrade/sample-v2  # This will be our next version of the service

7. Update YANG Module in sample-v2

  augment /ncs:services {
    list sample {
      description “This is an RFS skeleton service”;

      key name;
      leaf name {
        tailf:info “Unique service id”;
        tailf:cli-allow-range;
        type string;
      }

      uses ncs:service-data;
      ncs:servicepoint sample-servicepoint;

      // Define simple VRF parameters: Name and VRF-ID for route-target config

      // We have to keep the old nodes for the copy operation
      leaf vrf-name {
        type string;
      }

      leaf vrf-id {
        type uint16 {
          tailf:info “<0-65535>;;VRF-ID”;
          range “0..65535”;
        }
      }

      // This defines the target YANG structure
      container vrf {
        leaf vrf-name {
          type string;
        }

        leaf vrf-id {
          type uint16 {
            tailf:info “<0-65535>;;VRF-ID”;
            range “0..65535”;
          }
        }
      }
    }
  }

8. Print updated Service YANG Module as tree

$ pyang -f tree sample.yang
module: sample
augment /ncs:services:
   +--rw sample* [name]
      +--rw name                        string
      ...
      +--rw vrf-name?                   string
      +--rw vrf-id?                     uint16
      +--rw vrf
         +--rw vrf-name?   string
         +--rw vrf-id?     uint16

9. Write Python code to copy values from old to new YANG nodes

$ cat ~/dev/NSO/projects/python-upgrade/packages/upgrade/sample-v2/python/sample/upgrade.py

# -*- mode: python; python-indent: 4 -*-
import ncs
from ncs.application import Service

class Main(ncs.application.Application):
    def setup(self):
        self.log.info(‘Upgrade Main RUNNING’)
        with ncs.maapi.Maapi() as m:
            with ncs.maapi.Session(m, ‘admin’, ‘python’):
                with m.start_write_trans() as t:
                    root = ncs.maagic.get_root(t)
                    for sample in root.services.sample:
                        if sample.vrf.name is None:
                            self.log.info(‘Migrating sample instace with name=‘,sample.name)
                            sample.vrf.vrf_name = sample.vrf_name
                            sample.vrf.vrf_id = sample.vrf_id

                            sample.vrf = None
                            sample.vrf_id = None
                            self.log.info(‘Migration completed for sample instance ‘,sample.name)
                        else:
                            self.log.info(‘Service does not require migration: ‘,sample.name)

                    t.apply(keep_open=False)

    def teardown(self):
        self.log.info(‘Upgrade Main FINISHED’)

10. Update package-meta-data file to include new python upgrade class

$ cat ~/dev/NSO/projects/python-upgrade/packages/upgrade/sample-v2/package-meta-data.xml

<ncs-package xmlns=“http://tail-f.com/ns/ncs-packages”>
  <name>sample</name>
  <package-version>1.0</package-version>
  <description>Generated Python package</description>
  <ncs-min-version>4.4</ncs-min-version>

  <component>
    <name>main</name>
    <application>
      <python-class-name>sample.main.Main</python-class-name>
    </application>
  </component>
  <component>
    <name>upgrade</name>
    <application>
      <python-class-name>sample.upgrade.Main</python-class-name>
    </application>
  </component>
</ncs-package>

11. Replace existing sample service with upgraded sample service

$ rm -rf ~/dev/NSO/projects/python-upgrade/packages/sample
$ cp -R ~/dev/NSO/projects/python-upgrade/packages/upgrade/sample-v2 ~/dev/NSO/projects/python-upgrade/packages/sample

12. Import upgraded service into NSO

$ ncs_cli -u admin

admin connected from 127.0.0.1 using console on MLESKE-M-Q005
admin@ncs> show configuration services sample
sample 100 {
    vrf-name vrf100;
    vrf-id   100;
}
sample 101 {
    vrf-name vrf101;
    vrf-id   101;
}
admin@ncs> request packages reload
reload-result {
    package sample
    result true
}
[ok][2017-09-28 10:52:23]
admin@ncs>
System message at 2017-09-28 10:52:23...
Commit performed by admin via tcp using python.
admin@ncs>
admin@ncs> show configuration services sample
sample 100 {
    vrf {
        vrf-name vrf100;
        vrf-id   100;
    }
}
sample 101 {
    vrf {
        vrf-name vrf101;
        vrf-id   101;
    }
}

$ tail -f ncs-python-vm-sample.log
...
2017-09-28 10:52:23 - sample - INFO - Main FINISHED
2017-09-28 10:52:23 - sample - INFO - Exited
2017-09-28 10:52:23 - sample - INFO - Goodbye cruel world...
2017-09-28 10:52:23 - sample - INFO - Starting...
2017-09-28 10:52:23 - sample - INFO - Started
2017-09-28 10:52:23 - sample - INFO - Upgrade Main RUNNING
2017-09-28 10:52:23 - sample - INFO - Main RUNNING
2017-09-28 10:52:23 - sample - INFO - Migrating sample instace with name=100
2017-09-28 10:52:23 - sample - INFO - Migration completed for sample instance 100
2017-09-28 10:52:23 - sample - INFO - Migrating sample instace with name=101
2017-09-28 10:52:23 - sample - INFO - Migration completed for sample instance 101
2017-09-28 10:52:23 - sample - INFO - Service create(service=/ncs:services/sample:sample{100})
2017-09-28 10:52:23 - sample - INFO - Service create(service=/ncs:services/sample:sample{101})
...