How does one apply a service to multiple devices in one CLI (ex create Loopback interfaces and assign IPs to them)?

[Q5]

Hi guys

I am doing some light reading of the documentation and I am having issues to understand how does one apply a service to multiple devices in one CLI (ex create Loopback interfaces and assign IPs to them).

usually, you can apply a service to a device and pass an IP as parameter

How is it done when you have to do it on multiple devices? You could probably pass the device name as a group (members of that group) but then how do you generate the device name-IP pair for each device? Is this done using the the XML template built in logic ? Please share an example if you have one.
The only other ways I am seeing it would be via Python code or North bound APIs

[vleijon]

There are a number of options depending on what you actually want to do and how you want the addresses to be assigned. What I am describing here are different ways to achieve this in a service design, and this is something that the service designer would decide on design time.

First of all, if you want multiple devices with user-specified IPs you can create a list of device/ip pairs as input and loop over that list. This is perhaps the easiest solution to implement but puts the burden of coming up with the addresses to the users.

Second of all, you can give NSO a range of addresses and allocate from that range (a module called the “resource-manager”). Then all you have to do is specify the devices and let NSO automatically allocate a loopback address.

Finally, If neither of those is what you want you can implement whatever logic you want, such as calling an IPAM system. You can also imagine a mix of the first two methods where things are allocated if not explicitly specified.

A good example of this kind of things is examples.ncs/getting-started/developing-with-ncs/17-mpls-vpn-python where the services consists of lists of nodes, interfaces and addresses.

I hope at lest some of this was helpful.

[Q5]

That example is very complex and the many things done there tend to confuse the reader

I went over the read me which shows how the things are done and configured and everything seems to be passed manually, discrete values 

I am looking for a CLI option like 

ncs(config)# service loopback device "group-name-here" ip-range "ip-range-here"

but I am confused on how the template (XML) should look like and how it will generate configs for each device-name/IP pair
I am still reading these topics so please forgive my ignorance

[vleijon · Accepted solution]

I will see if I can make it clearer. The basic answer is that it depends on the service developer how it is implemented, there is no implicit abstraction since this is very service dependant.

What you can do is either do a pair-wise mapping where you give an explicit list of device/address pairs or you can do an explicit range, something like this: example ex1 devices group g1 start 192.168.1.1 stop 192.168.1.20.

The template language has control statements such as for and foreach that can be used to iterate over lists.

[Q5]

Pretty much what I was expecting..the only issue is that I am missing an example, I am a beginner with NSO

Unless I get some help here I will have to figure out how this is done via the template logic by extrapolating some of the examples available 

[lmanor · Accepted solution]

Actually, you really don't need to use any template procession instruction to do this.
The main idea here is that when the template engine hits a list (or leaf-list) it will try to process each entry int the list.  Please see chapter 11, Templates in the NSO Developer's guide.

Quick example:

yang$ cat add-loopback.yang

module add-loopback {

  namespace "http://com/example/addloopback";

  prefix add-loopback;

  import ietf-inet-types {

    prefix inet;

  }

  import tailf-ncs {

    prefix ncs;

  }

  list add-loopback {

    key name;

    uses ncs:service-data;

    ncs:servicepoint "add-loopback";

    leaf name {

      type string;

    }

    leaf-list device {

      type leafref {

        path "/ncs:devices/ncs:device/ncs:name";

      }

    }

    leaf intf-name {

        type string;

    }

    leaf intf-addr {

      type inet:ipv4-address;

    }

    leaf intf-mask {

      type inet:ipv4-address;

    }

  }

}

Template:

templates$ cat add-loopback-template.xml

<config-template xmlns="http://tail-f.com/ns/config/1.0"

                 servicepoint="add-loopback">

  <devices xmlns="http://tail-f.com/ns/ncs">

  <device>

    <name>{device}</name>

      <config tags='merge'>

      <interface xmlns="urn:ios">

      <Loopback>

        <name>{/intf-name}</name>

        <ip>

          <address>

            <primary>

              <address>{/intf-addr}</address>

              <mask>{/intf-mask}</mask>

            </primary>

          </address>

        </ip>

      </Loopback>

      </interface>

      </config>

  </device>

  </devices>

</config-template>

Running the service - note that the 'device' is a leaf-list, we use 2 devices:

admin@ncs% set add-loopback lb1 device [ ios-0 ios-1 ] intf-name 1 intf-addr 127.0.0.1 intf-mask 255.0.0.0

[ok][2019-09-23 14:24:58]

[edit]

admin@ncs% show add-loopback                                                        add-loopback lb1 {

    device    [ ios-0 ios-1 ];

    intf-name 1;

    intf-addr 127.0.0.1;

    intf-mask 255.0.0.0;

}

Can see on a commit dry-run that the loopback interface will get set on both devices as a result of this service:

admin@ncs% commit dry-run

cli {

    local-node {

        data  devices {

                  device ios-0 {

                      config {

                          ios:interface {

             +                Loopback 1 {

             +                    ip {

             +                        address {

             +                            primary {

             +                                address 127.0.0.1;

             +                                mask 255.0.0.0;

             +                            }

             +                        }

             +                    }

             +                }

                          }

                      }

                  }

                  device ios-1 {

                      config {

                          ios:interface {

             +                Loopback 1 {

             +                    ip {

             +                        address {

             +                            primary {

             +                                address 127.0.0.1;

             +                                mask 255.0.0.0;

             +                            }

             +                        }

             +                    }

             +                }

                          }

                      }

                  }

              }

             +add-loopback lb1 {

             +    device [ ios-0 ios-1 ];

             +    intf-name 1;

             +    intf-addr 127.0.0.1;

             +    intf-mask 255.0.0.0;

             +}

    }

}

Further, you can invoke the template debug feature to see exactly what the template engine is doing, it finds 2 devices listed in the service 'device' leaf-list and proceeds to process the subsequent template items for each device:

admin@ncs% commit dry-run | debug template

Evaluating "device" (from file "add-loopback-template.xml", line 5)

Context node: /add-loopback[name='lb1']

Result:

For /add-loopback[name='lb1']/device[.='ios-0'], it evaluates to "ios-0"

For /add-loopback[name='lb1']/device[.='ios-1'], it evaluates to "ios-1"

Operation 'merge' on existing node: /devices/device[name='ios-0'] (from file "add-loopback-template.xml", line 5)

Evaluating "/intf-name" (from file "add-loopback-template.xml", line 9)

Context node: /add-loopback[name='lb1']/device[.='ios-0']

Result:

For /add-loopback[name='lb1'], it evaluates to "1"

Operation 'merge' on non-existing node: /devices/device[name='ios-0']/config/ios:interface/Loopback[name='1'] (from file "add-loopback-template.xml", line 9)

Operation 'merge' on non-existing node: /devices/device[name='ios-0']/config/ios:interface/Loopback[name='1']/ip/address/primary/address (from file "add-loopback-template.xml", line 13)

Evaluating "/intf-addr" (from file "add-loopback-template.xml", line 13)

Context node: /add-loopback[name='lb1']

Result:

For /add-loopback[name='lb1'], it evaluates to "127.0.0.1"

Setting /devices/device[name='ios-0']/config/ios:interface/Loopback[name='1']/ip/address/primary/address to "127.0.0.1"

Operation 'merge' on non-existing node: /devices/device[name='ios-0']/config/ios:interface/Loopback[name='1']/ip/address/primary/mask (from file "add-loopback-template.xml", line 14)

Evaluating "/intf-mask" (from file "add-loopback-template.xml", line 14)

Context node: /add-loopback[name='lb1']

Result:

For /add-loopback[name='lb1'], it evaluates to "255.0.0.0"

Setting /devices/device[name='ios-0']/config/ios:interface/Loopback[name='1']/ip/address/primary/mask to "255.0.0.0"

Operation 'merge' on existing node: /devices/device[name='ios-1'] (from file "add-loopback-template.xml", line 5)

Evaluating "/intf-name" (from file "add-loopback-template.xml", line 9)

Context node: /add-loopback[name='lb1']/device[.='ios-1']

Result:

For /add-loopback[name='lb1'], it evaluates to "1"

Operation 'merge' on non-existing node: /devices/device[name='ios-1']/config/ios:interface/Loopback[name='1'] (from file "add-loopback-template.xml", line 9)

Operation 'merge' on non-existing node: /devices/device[name='ios-1']/config/ios:interface/Loopback[name='1']/ip/address/primary/address (from file "add-loopback-template.xml", line 13)

Evaluating "/intf-addr" (from file "add-loopback-template.xml", line 13)

Context node: /add-loopback[name='lb1']

Result:

For /add-loopback[name='lb1'], it evaluates to "127.0.0.1"

Setting /devices/device[name='ios-1']/config/ios:interface/Loopback[name='1']/ip/address/primary/address to "127.0.0.1"

Operation 'merge' on non-existing node: /devices/device[name='ios-1']/config/ios:interface/Loopback[name='1']/ip/address/primary/mask (from file "add-loopback-template.xml", line 14)

Evaluating "/intf-mask" (from file "add-loopback-template.xml", line 14)

Context node: /add-loopback[name='lb1']

Result:

For /add-loopback[name='lb1'], it evaluates to "255.0.0.0"

Setting /devices/device[name='ios-1']/config/ios:interface/Loopback[name='1']/ip/address/primary/mask to "255.0.0.0"

[Q5]

Thanks Imanor that was almost exactly what I was looking for excepting one thing: I want or I need to assign different IPs for the loopback interface I am creating on each device, like so:
R1 Loopback1  192.168.1.1

R2 Loopback1  192.168.1.2

R3 Loopback1  192.168.1.3

So the command like would be like

set add-loopback lb1 device [ R1 R2 R3 ] intf-name 1 startip 192.168.1.1  endip 192.168.1.3

and I am curious to know what would be the template logic for the above in order to pair each router with one IP. 

[lmanor · Accepted solution]

Right,

If you want a specific address for a device you can always model it like this a provide the explicit ipaddr and mask for every device - which would allow for the most flexible address/mask variations:

    list add-lb-per-device {

      key device;

      leaf device {

        type leafref {

          path "/ncs:devices/ncs:device/ncs:name";

        }

      }

      leaf intf-name {

          type string;

      }

      leaf intf-addr {

        type inet:ipv4-address;

      }

      leaf intf-mask {

        type inet:ipv4-address;

      }

    }

and

admin@ncs% set add-loopback lb1 add-lb-per-device ios-0 intf-name 1 intf-addr 127.0.0.1 intf-mask 255.0.0.0

admin@ncs% set add-loopback lb1 add-lb-per-device ios-1 intf-name 1 intf-addr 127.0.0.2 intf-mask 255.0.0.0

admin@ncs% show add-loopback lb1

add-lb-per-device ios-0 {

    intf-name 1;

    intf-addr 127.0.0.1;

    intf-mask 255.0.0.0;

}

add-lb-per-device ios-1 {

    intf-name 1;

    intf-addr 127.0.0.2;

    intf-mask 255.0.0.0;

}

and a simple template like this:

  <device>

    <name>{/add-lb-per-device/device}</name>

      <config tags='merge'>

      <interface xmlns="urn:ios">

      <Loopback>

        <name>{intf-name}</name>

        <ip>

          <address>

            <primary>

              <address>{intf-addr}</address>

              <mask>{intf-mask}</mask>

            </primary>

          </address>

        </ip>

      </Loopback>

      </interface>

      </config>

  </device>

However, if you want to employ some scheme that each address is incremental to the prior for each subsequent device, you would need to use some of the template processing commands to deal with string replacement and host incrementing, something like this:

Yang:

  list add-loopback {

    key name;

    uses ncs:service-data;

    ncs:servicepoint "add-loopback";

    leaf name {

      type string;

    }

    leaf-list device {

      type leafref {

        path "/ncs:devices/ncs:device/ncs:name";

      }

    }

    leaf intf-name {

        type string;

    }

    leaf intf-addr {

      type inet:ipv4-address;

    }

    leaf intf-addr-incement {

      type uint8;

    }

    leaf intf-mask {

      type inet:ipv4-address;

    }

  }

Template:

<config-template xmlns="http://tail-f.com/ns/config/1.0"

                 servicepoint="add-loopback">

  <devices xmlns="http://tail-f.com/ns/ncs">

  <?set ipaddr = {/intf-addr}?>

  <?set len={string-length($ipaddr)}?>

  <?set ipnetwork = {substring($ipaddr, 0, $len - 1)?>

  <?set iphost = {substring($ipaddr, $len, $len)?>

  <?set inc = {/intf-addr-incement}?>

  <device>

    <name>{device}</name>

      <config tags='merge'>

      <interface xmlns="urn:ios">

      <Loopback>

        <name>{/intf-name}</name>

        <ip>

          <address>

            <primary>

              <?set iphost = {$iphost + $inc}?>

              <address>{$ipnetwork}{$iphost}</address>

              <mask>{/intf-mask}</mask>

            </primary>

          </address>

        </ip>

      </Loopback>

      </interface>

      </config>

  </device>

  </devices>

</config-template>

admin@ncs% set add-loopback lb1 device [ ios-0 ios-1 ] intf-name 1 intf-addr 127.0.0.0 intf-mask 255.0.0.0 intf-addr-incement 1

[ok][2019-09-24 12:00:17]

admin@ncs% show add-loopback

add-loopback lb1 {

    device             [ ios-0 ios-1 ];

    intf-name          1;

    intf-addr          127.0.0.0;

    intf-addr-incement 1;

    intf-mask          255.0.0.0;

}

admin@ncs% commit dry-run

cli {

    local-node {

        data  devices {

                  device ios-0 {

                      config {

                          ios:interface {

             +                Loopback 1 {

             +                    ip {

             +                        address {

             +                            primary {

             +                                address 127.0.0.1;

             +                                mask 255.0.0.0;

             +                            }

             +                        }

             +                    }

             +                }

                          }

                      }

                  }

                  device ios-1 {

                      config {

                          ios:interface {

             +                Loopback 1 {

             +                    ip {

             +                        address {

             +                            primary {

             +                                address 127.0.0.2;

             +                                mask 255.0.0.0;

             +                            }

             +                        }

             +                    }

             +                }

                          }

                      }

                  }

              }

             +add-loopback lb1 {

             +    device [ ios-0 ios-1 ];

             +    intf-name 1;

             +    intf-addr 127.0.0.0;

             +    intf-mask 255.0.0.0;

             +}

    }

}

[Q5]

You hit the nail, that is what I was looking for, thanks!
Related to this, I am using REST APIs to send this:
<config>
<loopback xmlns="http://com/example/loopback">
<servicename>R1Loopback</servicename>
<device>R1</device>
<loopback-intf-name>1111</loopback-intf-name>
<ip-address>10.100.1.1</ip-address>
</loopback>
<loopback xmlns="http://com/example/loopback">
<servicename>R2Loopback</servicename>
<device>R2</device>
<loopback-intf-name>1112</loopback-intf-name>
<ip-address>10.100.1.2</ip-address>
</loopback>
<loopback xmlns="http://com/example/loopback">
<servicename>R3Loopback</servicename>
<device>R3</device>
<loopback-intf-name>1113</loopback-intf-name>
<ip-address>10.100.1.3</ip-address>
</loopback>
<loopback xmlns="http://com/example/loopback">
<servicename>R4Loopback</servicename>
<device>R4</device>
<loopback-intf-name>1114</loopback-intf-name>
<ip-address>10.100.1.4</ip-address>
</loopback>
<loopback xmlns="http://com/example/loopback">
<servicename>R5Loopback</servicename>
<device>R5</device>
<loopback-intf-name>1115</loopback-intf-name>
<ip-address>10.100.1.5</ip-address>
</loopback>
<loopback xmlns="http://com/example/loopback">
<servicename>R6Loopback</servicename>
<device>R6</device>
<loopback-intf-name>1116</loopback-intf-name>
<ip-address>10.100.1.6</ip-address>
</loopback>
</config>

Why is it taking like 20 minutes to calculate config diffs, log below.
If I do the same using CLI (create 6 services and then commit) I am seeing the same slowness. Any idea how I can troubleshoot this ?

<DEBUG> 23-Sep-2019::06:46:01.913 NSOComputer ncs[20064]: ncs Requestor {commit,246,<0.204.0>} tries to acquire lock for device <<"R3">>
<DEBUG> 23-Sep-2019::06:46:01.919 NSOComputer ncs[20064]: ncs Requestor {commit,246,<0.204.0>} tries to acquire lock for device <<"R4">>
<DEBUG> 23-Sep-2019::06:46:01.920 NSOComputer ncs[20064]: ncs Requestor {commit,246,<0.204.0>} tries to acquire lock for device <<"R5">>
<DEBUG> 23-Sep-2019::06:46:01.920 NSOComputer ncs[20064]: ncs Requestor {commit,246,<0.204.0>} tries to acquire lock for device <<"R6">>
<DEBUG> 23-Sep-2019::06:46:01.920 NSOComputer ncs[20064]: ncs Requestor {commit,246,<0.204.0>} tries to acquire lock for device <<"R1">>
<DEBUG> 23-Sep-2019::06:46:01.921 NSOComputer ncs[20064]: ncs Requestor {commit,246,<0.204.0>} tries to acquire lock for device <<"R2">>
<DEBUG> 23-Sep-2019::06:46:01.922 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R3: calculating southbound diff...
<DEBUG> 23-Sep-2019::06:46:01.922 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R4: calculating southbound diff...
<DEBUG> 23-Sep-2019::06:46:01.922 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R5: calculating southbound diff...
<DEBUG> 23-Sep-2019::06:46:01.922 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R6: calculating southbound diff...
<DEBUG> 23-Sep-2019::06:46:01.923 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R1: calculating southbound diff...
<DEBUG> 23-Sep-2019::06:46:01.923 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R2: calculating southbound diff...






<DEBUG> 23-Sep-2019::07:07:33.274 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R1: calculating southbound diff ok
<DEBUG> 23-Sep-2019::07:07:35.235 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R3: calculating southbound diff ok
<DEBUG> 23-Sep-2019::07:07:35.279 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R6: calculating southbound diff ok
<DEBUG> 23-Sep-2019::07:07:40.633 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R4: calculating southbound diff ok
<DEBUG> 23-Sep-2019::07:07:40.684 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R2: calculating southbound diff ok
<DEBUG> 23-Sep-2019::07:07:42.929 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R5: calculating southbound diff ok
<DEBUG> 23-Sep-2019::07:07:42.930 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R3: connect: device connect...
<DEBUG> 23-Sep-2019::07:07:42.932 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R4: connect: device connect...
<DEBUG> 23-Sep-2019::07:07:42.933 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R5: connect: device connect...
<DEBUG> 23-Sep-2019::07:07:42.938 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R6: connect: device connect...
<DEBUG> 23-Sep-2019::07:07:42.938 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R1: connect: device connect...
<DEBUG> 23-Sep-2019::07:07:42.938 NSOComputer ncs[20064]: ncs commit progress db=running usid=246 thandle=3482: ncs: device R2: connect: device connect...
<DEBUG> 23-Sep-2019::07:07:42.943 NSOComputer ncs[20064]: devel-cdb connect from Cdb-ResourceManaged-41