This is a variation of the "Using NSO actions to check operational state with pyATS parsers" blog post I've written but with another final objective, instead of generating an Excel document we will generate a pytest report.
In this post we will see how to retrieve the checks generated by NSO action to build the fixtures and give it to the test function. As it is based on the previous post, we will do the same thing check if the version of the device is the same as the target version we will give as parameter.
The code is in this repository : https://github.com/rtrjl/pytest_network
Install pytest :
pip install pytest
Assuming you have a working NSO with the package "check_device" fully functional you are able to retrieve the checks via the RESTConf API.
The test function is very simple :
def test_router_version(device_check):
assert device_check["check_status"] == CHECK_OK
The difficult task here is to automatically put something in the parameter "device_check" : the fixture which is the result retrieved from the NSO action "check_version".
Let's see the code , it is almost the same as the one you have seen to generate the Excel document but with more error handling :
from concurrent.futures import ThreadPoolExecutor, as_completed
import json
from nso_restconf.restconf import RestConf
# don't put your credentials in the code use ENV variables instead
nso_api = RestConf(address="http://127.0.0.1", port=8080,
username="admin", password="admin")
CHECK_OK = "OK"
CHECK_NOK = "NOK"
CHECK_ERROR = "ERROR"
def router_version_check_data(target_version, devices_list) -> list:
processes = []
with ThreadPoolExecutor(max_workers=5) as executor:
for device in devices_list:
processes.append(executor.submit(nso_check_version, device, target_version))
checks_list = []
for task in as_completed(processes):
result = task.result()["check_device:output"]
checks_list.append(result)
return checks_list
def nso_check_version(device, target_version):
try:
data = json.dumps({'input': {'device': f"{device}", "target_version": f"{target_version}"}})
result = nso_api.action(data, "check_device/check_version")
return result.json()
except ConnectionError:
return {"check_device:output": {"device": device, "check_status": CHECK_NOK,
"check_message": f"Cannot connect to NSO to check {device}"}}
As you see we also retrieve the results in parallel with the concurrent.futures .
Ok but how the "device_check" parameter of the test is generated ?
There is an optional file "conftest.py" that can be used to build dynamically the fixtures you need for your tests.
This file , and the "specials" functions inside, the hooks, will be called as pytest is launched, and if some hooks exists they will be also called (in an order predefined by pytest developers)
def pytest_addoption(parser):
parser.addoption("--target-version", action="store", help="target version of the devices")
parser.addoption("--devices-list-path", action="store", help="path to the devices list txt")
In this part of the conftest.py the hook "pytest_addoption" is used to ... add options.
We need to give to the pytest command line the target version we want to test and the devices list.
def pytest_generate_tests(metafunc):
if "device_check" in metafunc.fixturenames:
target_version = metafunc.config.getoption("--target-version", default=None)
devices_list_path = metafunc.config.getoption("--devices-list-path", default=None)
if devices_list_path:
with open(devices_list_path) as devices_list_txt:
devices_list = devices_list_txt.readlines()
devices_list = [str(device).strip() for device in devices_list]
if target_version and devices_list_path:
check_list = router_version_check_data(target_version, devices_list)
metafunc.parametrize("device_check", check_list)
else:
metafunc.parametrize("device_check", [])
Here is the core of our fixtures generator , this hook "pytest_generate_tests" will call the "router_version_check_data" function, retrieve the data from NSO and give it to our test function "test_router_version".
This hook is called any time pytest see a test function so we need to put "if "device_check" in metafunc.fixturenames:", to run the NSO actions only if the test need it.
pytest -v --target-version="7.1.15" --devices-list-path="devices_list.txt"
=========================================================================== short test summary info ===========================================================================
FAILED test_version_with_pytest.py::test_router_version[device_check0] - AssertionError: assert 'NOK' == 'OK' for Paris-66
FAILED test_version_with_pytest.py::test_router_version[device_check1] - AssertionError: assert 'NOK' == 'OK' for Kiev-47
FAILED test_version_with_pytest.py::test_router_version[device_check2] - AssertionError: assert 'NOK' == 'OK' for Riga-46
FAILED test_version_with_pytest.py::test_router_version[device_check3] - AssertionError: assert 'NOK' == 'OK' for Stuttgard-64
FAILED test_version_with_pytest.py::test_router_version[device_check4] - AssertionError: assert 'NOK' == 'OK' for Napoli-50
FAILED test_version_with_pytest.py::test_router_version[device_check5] - AssertionError: assert 'NOK' == 'OK' for Madrid-40
FAILED test_version_with_pytest.py::test_dummy_test_will_fail - assert False
========================================================================= 7 failed, 1 passed in 2.05s =========================================================================
That's all, with less of 60 lines of code we have a powerful automated tool to check the version of devices and produce a network audit. Pytest can be integrated with a lot of other tools (Jenkins, Gitlab ...) and the limit is your imagination.
(At the end of the conftest.py, I've added a trick to customize the summary output of pytest to add the devices name at the end of the lines feel free to adapt to your needs.)
