For those of you who are using Python for you code, knows that there is only one way to be sure and it is to run the code. But how do you know what parts of you code have been executed?! I do very often put in print statements and look at the output and very often I find that I need to add more statements and to do that I need to restart the package. This time consuming and boring and thought I should share my experiences how to mitigate this problem.
During some other test related work I came across Coverage.py and it was much easier to use than I thought and can produce HTML reports that gives a very good overview and also details down to each line of code loaded to memory.
Here is one example to how to use it for a simple Python based service. The package name is python_service_template and will be run outside of NSO.
Installation of coverage:
pip3 install coverage
First you need to be able to stop the Python VM:s started by NSO. Add this to ncs.conf and reload or restart NSO:
<hide-group>
<name>debug</name>
</hide-group>
Stop the running python-vm in the NSO cli:
admin@ncs# unhide debug
admin@ncs# python-vm stop name python_service_template
result OK
Start the service's Python VM on the command line. This has beed tested with a NSO local installation:
coverage run /Users/uforsgre/ncs-release/5.4.1/src/ncs/pyapi/ncs_pyvm/startup.py -l debug -i python_service_template
Now you will do your tests, creating services, modifing services, etc.
When you are done and ready to get truth, you stop the Python VM, again:
admin@ncs# python-vm stop name python_service_template
result OK
I am not sure if it works just to press ctrl-c in the terminal windows where service is running. I leave that to you to find out.
Now you will find a file .coverage in the directory where the service was running. To show a textual overview just run:
coverage report
To produce a more extensive HTML report run:
coverage html
The HTML report end up in a directory htmlcov and the overview looks like this, showing a summary of all used modules. It also has a seach function if you have a lot of files.
Click on any of the files will show a detailed view for that file. Here you can highlight lines the have been executed:
This is a view some lines highlighted as not executed:
Which I personally think is a good way of getting input to create tests to improve the coverage of my tests. Since I had 100% coverage in my service I am showing this part the application class in the NSO API, revealing that I have no actions in my service.
Let me know if you find this useful and have ideas how we can impore development of packages, services and other script for NSO.
Ulrik Stridsman
Software Engineer
NSO CTO Team