How do I call the 'devices device foo config exec ...' action from Python?

[allenc]

I am writing a Python action such that I can run certain commands from Python. For example, I would like to be able to do the equivalent of this CLI command from Python:

devices device csr1 config exec "ip tcp window-size ?"

This command works fine from the CLI, but I have not been able to figure out how to do it from Python. I tried implementing it using the same pattern that I use for executing command from live-status, but that doesn't work. Here is a snippet showing what I am trying to do:

device_keypath = f'/devices/device{{{device_name}}}'
device = ncs.maagic.get_node(self.read_transaction, device_keypath)
if mode == 'operational':
    exe = device.live_status.ios_stats__exec.any
elif mode == 'configure':
    exe = device.config.exec
else:
    raise ValueError(
        "Invalid command mode. Must be 'operational' or "
        "'configure'")
input = exe.get_input()
input.args = [command]
output = exe.request(input)

The above code works in "operational" mode, which uses `live-status exec any`. However, the "configure" mode does not work.

Here is the exception I get: 

AttributeError: 'Container' object has no attribute 'get_input'

In trying to troubleshoot this, I did a dir() on device.config.exec and see the following:

['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', 
'__format__', '__ge__', '__get__', '__getattribute__', '__gt__', 
'__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', 
'__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', 
'__self__', '__self_class__', '__setattr__', '__sizeof__', '__str__', 
'__subclasshook__', '__thisclass__', 'any', 'execute', 'get', 'show', 
'system_ha_config', 'vdom_admin']

This does not mirror what I see in the CLI (for devices using the cisco-ios NED). In the CLI, it appears that device.config.exec is an action, but from Maagic, it looks like it is a container that contains other actions (any, execute, get, ...). I tried `any` and `execute` but they both seem to run commands in operational mode, not configure mode.  I am confused why this does not match the CLI.

Can someone tell me how to call `config exec` from Python?  Bonus points if you can also explain why device.config.exec above does not match what you see from the CLI.

Thanks,
-Allen

[gschudel · Accepted solution]

Hi Allen

** i got it to work... see below...

** but, i'm wondering _why_ you wish to do this? (I can't come up w/ a good reason... :)

First i looked at the device behavior:

this is a cisco-ios behavior from "config mode" -- not the normal "show" output stuff... E.g.

CSR#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
CSR(config)#
CSR(config)#ip tcp window-size ?
<0-1073741823> Window size

CSR(config)#

Then, I looked up the YANG stuff in the cisco-ios NED to get an idea of where this was located and how to get at it... 

/// ========================================================================
/// EXEC (exec commands executed in config mode)
/// ========================================================================

container EXEC {
tailf:cli-drop-node-name;

// default
tailf:action "default" {
tailf:info "Set a command to its defaults";
tailf:actionpoint ncsinternal {
tailf:internal;
}
input {
list auto-prompts {
tailf:cli-suppress-mode;
key question;
leaf question {
type string;
}
leaf answer {
type string;
}
}
leaf-list args {
tailf:cli-drop-node-name;
tailf:cli-flat-list-syntax;
type string {
tailf:info "ACTION:;;default <argument(s)>";
}
}
}
output {
leaf result {
type string;
}
}
}

Then i ran a few commands from NSO CLI to further gain ideas. E.g.: 

(1) "live-status" example

admin@ncs% request devices device csr1 live-status ios-stats:exec any args "show ip int brief"
result
> show ip int brie
Interface IP-Address OK? Method Status Protocol
GigabitEthernet1 10.87.163.43 YES DHCP up up
GigabitEthernet2 192.105.1.11 YES DHCP up up
CSR#
[ok][2018-11-20 23:54:04]

[edit]
admin@ncs%

(2) "EXEC" example

(config mode)

admin@ncs% request devices device csr1 config ios:EXEC exec args "ip tcp window-size ?"
result
> ip tcp window-size ?
<0-1073741823> Window size

CSR(config)#ip tcp window-size
[ok][2018-11-20 20:28:20]

[edit]
admin@ncs%

(operation mode)

admin@ncs> request devices device csr1 config ios:EXEC exec args "ip tcp window-size ?"
result
> ip tcp window-size ?
<0-1073741823> Window size

CSR(config)#ip tcp window-size
[ok][2018-11-20 20:29:41]
admin@ncs>

OK! that all works...

Now to python action...  I have two calls here -- BOTH of the ones above -- i.e. the live-status one and the EXEC one...

(** ignore my style.. it's brute force and lot's of debug/comments)...

[root@studentnso iptcpwin]# cat python/iptcpwin/main.py
# -*- mode: python; python-indent: 4 -*-
import ncs
import _ncs
from ncs.dp import Action
import os
from ncs import maagic
from ncs import maapi

# ---------------
# ACTIONS EXAMPLE
# ---------------
class Doiptcpwin(Action):
    @Action.action
    def cb_action(self, uinfo, name, kp, input, output):
        self.log.info('action name: ', name)
        self.log.info('action input.device: ', input.device)

        #...............................................................................
        port = int(os.getenv('NCS_IPC_PORT', _ncs.NCS_PORT))
        try:
            with maapi.Maapi("127.0.0.1", port, path=None) as self.maapi:
                self.maapi.start_user_session("admin", "system" , [])
                with self.maapi.start_write_trans(db=ncs.RUNNING) as t:
                    self.log.debug(" Transaction t :: [%s]" % (t))
                    root = maagic.get_root(t, shared=False) 
                    self.log.debug(" Root :: [%s]" % (root))
                    #
                    #..............................................
                    newdev = root.devices.device[input.device]
                    self.log.debug(" thisdevice name :: [%s]" % (newdev.name))
                    self.log.debug(" thisdevice type :: [%s]" % (newdev.device_type.ne_type))
                    self.log.debug(" thisdevice ned :: [%s]" % (newdev.device_type.cli.ned_id))
                    #..............................................
                    #... item number 1 ............................
                    cmnd = 'any'
                    argm = 'show ip int brief'
                    inputArgs = argm.split(' ')
                    xinput = newdev.live_status.ios_stats__exec[cmnd].get_input()
                    xinput.args = inputArgs
                    otp = newdev.live_status.ios_stats__exec[cmnd](xinput)

                    print 'input :: %s' % (inputArgs)
                    print 'otp :: %s' % (otp)
                    print 'otp.result :: %s' % (otp.result)

                    output.result = '\n======================================================='
                    output.result += otp.result
                    output.result += '\n======================================================='

                    #... item number 2 ............................
                    cmnd = 'exec'
                    argm = 'ip tcp window-size ?'
                    yinputArgs = argm.split(' ')

                    yinput = newdev.config.ios__EXEC[cmnd].get_input()
                    yinput.args = yinputArgs
                    otp2 = newdev.config.ios__EXEC[cmnd](yinput)

                    print 'yinput :: %s' % (yinputArgs)
                    print 'otp2 :: %s' % (otp2)
                    print 'otp2.result :: %s' % (otp2.result)

                    output.result += '\n======================================================='
                    output.result += otp2.result
                    output.result += '\n======================================================='

        except Exception as e:
             output.result = "Exception during self-test action" 
             self.log.error(e)

        return ncs.CONFD_OK

# ---------------------------------------------
# COMPONENT THREAD THAT WILL BE STARTED BY NCS.
# ---------------------------------------------
class Main(ncs.application.Application):
    def setup(self):
        self.log.info('Main RUNNING')
        self.register_action('iptcpwin-action', Doiptcpwin)

    def teardown(self):
        self.log.info('Main FINISHED')
[root@studentnso iptcpwin]#

And the corresponding YANG...

[root@studentnso iptcpwin]# cat src/yang/iptcpwin.yang
module iptcpwin {

namespace "http://example.com/iptcpwin";
prefix iptcpwin;

import ietf-inet-types { prefix inet; }
import tailf-common { prefix tailf; }
import tailf-ncs { prefix ncs; }

container doexec {
tailf:action iptcpwin {
tailf:actionpoint iptcpwin-action;
input {
leaf device {
type string;
}
}
output {
leaf result {
type string;
}
}
}
}

}
[root@studentnso iptcpwin]#

Now to run this...

In the NSO CLI...

admin@ncs% request doexec iptcpwin device csr1
result
=======================================================
> show ip int brief
Interface IP-Address OK? Method Status Protocol
GigabitEthernet1 10.87.163.43 YES DHCP up up
GigabitEthernet2 192.105.1.11 YES DHCP up up
CSR#
=======================================================
=======================================================
> ip tcp window-size ?
<0-1073741823> Window size

CSR(config)#ip tcp window-size
=======================================================
[ok][2018-11-20 23:44:57]

[edit]
admin@ncs%

and from my log file...

[root@studentnso logs]# tail -f ncs-python-vm-iptcpwin.log

<DEBUG> 20-Nov-2018::23:44:55.299 iptcpwin ncs-dp-24850-iptcpwin:main: - _WsockCb.create_item() = <ncs.dp._Wsock object at 0x7fea68163fd0>
<DEBUG> 20-Nov-2018::23:44:55.299 iptcpwin ncs-dp-24850-iptcpwin:main-1: - ThreadPool._WorkerThread started <_WorkerThread(ncs-dp-24850-iptcpwin:main-1, started 140644324464384)>
<DEBUG> 20-Nov-2018::23:44:55.299 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - _WorkerThread running <_WorkerThread(ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action, started 140644324464384)>, item ncs-dp-24850-iptcpwin:main-1
<INFO> 20-Nov-2018::23:44:55.302 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - action name: iptcpwin
<INFO> 20-Nov-2018::23:44:55.302 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - action input.device: csr1
<DEBUG> 20-Nov-2018::23:44:55.305 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - Transaction t :: [Transaction th=8550]
<DEBUG> 20-Nov-2018::23:44:55.305 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - Root :: [(root)]
<DEBUG> 20-Nov-2018::23:44:55.308 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - thisdevice name :: [csr1]
<DEBUG> 20-Nov-2018::23:44:55.308 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - thisdevice type :: [cli]
<DEBUG> 20-Nov-2018::23:44:55.309 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - thisdevice ned :: [ios-id:cisco-ios]
<DEBUG> 20-Nov-2018::23:44:57.271 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - _WorkerThread finished <_WorkerThread(ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action, started 140644324464384)>, item ncs-dp-24850-iptcpwin:main-1
<DEBUG> 20-Nov-2018::23:44:57.271 iptcpwin ncs-dp-24850-iptcpwin:main-1-usid-1043-iptcpwin-action: - Pool(ncs-dp-24850-iptcpwin:main) idle: 0 busy: 0

Hopefully that helps...

** still interested in the "use-case" for this... :)

Thanks

Gregg

[allenc]

Hi Gregg,

Thank you very much! I have it working now thanks to your help. I really appreciate that you explained how you found the "path" by examining the YANG, too. Now I understand why the CLI doesn't match the Python path.

Here is some background to explain our use case.  We are deploying NSO in a brownfield enterprise with a wide variety of device models and OS versions. Unfortunately, the customer does not have a lab environment that has a representative sample of the production devices.

One challenge we are running into is that the services we build fail to deploy to some devices because those devices simply do not support all the configuration commands we are pushing. In some cases, the command isn't supported due to hardware limitations. In other cases, the command isn't supported in the version of IOS running on the device.

So, we are building this action to allow us to test commands against multiple devices and collect the outputs along with some metadata in order to understand what commands are supported on which devices and OS versions. We can then use this information to add logic to deal with exceptions in the services we develop.

If there is a better way to deal with these issues, we would love to hear it!

Thanks,

-Allen

[gschudel]

HI Allen - 

thank you for the background...

** in the cases you describe, i guess i would consider that the NED is not handling the specific device cases properly. The NED is expected to handle the device types and OS versions correctly -- so as guidance, i think at a minimum i'd suggest when you run into specific issues - open a TAC case against the NED - provide all the details: NSO version, NED version, device type and OS version... and the commands that are incorrectly handled... 

** you may also know about "trace raw" (on a per-device basis might be best) - and when you "sync-from" on the device (n the lab :) -- you'll see the raw interaction between NSO and the device -- with indications of failed commands...  this output also helps the TAC notes...

best regards

gregg

[allenc]

Hi Gregg,

That (the NEDs are not handling specific device cases properly) was our original position, too.  However, in multiple conversations with multiple parties at Cisco/Tail-F (including with developers at Developer Days earlier this year), we were told that's not how NEDs are meant to work and that it is up to us service developers to know what commands are supported by which devices... :(

-Allen

[gschudel]

interesting...

i've got a new task now... :)

[allenc]

Go get 'em! :)  Feel free to reach out if you want some specific cases.

[parkuma]

Hi Gregg, working with python action to execute live-status into admin mode:

devices device asr9k live-status cisco-ios-xr-stats:exec any admin-mode "show version" 

implement the same kind of pattern that as your python call to execute command from live-status, working fine for simple live-status arguments but that doesn't work for admin commands. Here is my class pattern

class LiveStatus(Action):
    @Action.action
    def cb_action(self, uinfo, name, kp, input, output, trans):
        ai = ncs.maapi.Maapi().get_authorization_info(uinfo.usid)
        with ncs.maapi.single_read_trans(uinfo.username, 'device-action', groups=[g for g in ai.groups], db=ncs.RUNNING) as t:

            device_name = input.device_name
            command = input.command
            arguments = input.arguments
            root = ncs.maagic.get_root(t)
            device = root.ncs__devices.device[device_name]
            input_args = arguments.split(' ')
            error_string = ''
            output1 = None
            self.log.info('device.module : ' + str(dir(device.module)))
            for module in device.module:
            #Try/catch so that it continues with all the tests even though one device is down.
                try:
                    self.log.info('module_name : ' + module.name)
                    if module.name == 'tailf-ned-cisco-ios':
                        self.log.info(device_name, ' is a cisco-ios device')
                        action_input = device.live_status.ios_stats__exec[command].get_input()
                        action_input.args = input_args
                        output1 = device.live_status.ios_stats__exec[command](action_input)
                        self.log.info('output: ' + str(dir(output1)))
                    elif module.name == 'tailf-ned-cisco-ios-xr':
                        self.log.info(device_name, ' is a cisco-iosxr device')
                        action_input = device.live_status.cisco_ios_xr_stats__exec[command].get_input()bcmvk4
                        action_input.args = input_args
                        output1 = device.live_status.cisco_ios_xr_stats__exec[command](action_input)
                        self.log.info('output: ' + str(dir(output1)))

And the followed yang-input

input {
        leaf device-name {
          type leafref{
            path "/ncs:devices/ncs:device/ncs:name";
          }
          mandatory true;
        }
        
        leaf command{
          type string;
          mandatory true;
        }

        leaf arguments{
          type string;
          mandatory true;
        }
      }

Please suggest here the possibilities to pass command/argument attributes within admin-mode.