cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
13764
Views
45
Helpful
11
Comments
Bruno De Oliveira Novais
Cisco Employee
Cisco Employee

Introduction

While gNMI is fairly new, it's becoming more and more powerful. Its abilities to simplify network management by the use of protocol buffer files and standard definitions are enabling our customers to integrate a lot better in multi-vendor environments.

 

It's also becoming more common to see aspiring developers (aka network engineers who know some coding) dealing with responsibilities of automating networks previously managed either manually or via old-fashioned scripts. It may seem a bit difficult to understand gNMI at first for a network engineer (or not if you're too smart!).

 

In this document I will go over the basics of setting up the stage for gNMI, as well as showing practical and working code as an example. Hopefully it simplifies the lives of these engineers, as well as showing the power of gNMI on IOS-XR.

 

This is not intended to show you how to build scaled software to support thousands of routers. It is instead going to help you get started. Once the building blocks are done, you can scale this as much as you want with the knowledge you have!

 

You might ask, why would I want to use gNMI at all when netconf is out there? The main reason is performance. By compressing data in binary format, gRPC can transfer a lot more data than xml, which netconf uses. About 3x - 10x faster. No big deal when you're dealing with 1 router, or maybe even hundreds. But can you manage hundreds of thousands of routers by transferring huge amounts of data all the time from them? Probably not!

Also, say you got some Python engineers, but tomorrow you hire some Go engineers. Guess what...the code is virtually the same between them - the libraries that we compile work the same way whether you use python, go, c++, etc. You don't have to build api's from scratch to manage a device, the proto buffer file is compiled into code for your use right away.

 

Take a look at the table below as well to get some more comparisons to similar methods:

 

Features netconf cisco grpc openconfig gnmi
Transport ssh http2 http2
Support vendor neutral proprietary vendor neutral
Encoding xml proto buff proto buff

gRPC Terminology and Installation

In practical terms, the gRPC Stub will be your *nix server. The IOS-XR router will be the gRPC Server. See below image for some clarity:grpc.png

 

Image from https://www.grpc.io/docs/guides/

 

Your proto file contains all the definitions of how the router in this case will receive requests and how to reply. These definitions are defined here: https://github.com/openconfig/gnmi/blob/master/proto/gnmi/gnmi.proto

 

For Python, all the instructions to install the gRPC tools needed are in the following link: https://www.grpc.io/docs/quickstart/python/

It's self-explanatory, but simple steps are:

 

1. Install grpcio

2. Install grpc-io tools - that includes the protoc, the compiler of the protocol buffer files.

3. run compiler

 

# create a virtual environment

$ virtualenv venv-grpc Using base prefix '/Library/Frameworks/Python.framework/Versions/3.7' New python executable in /Users/brusilva/venv-grpc/bin/python3.7 Also creating executable in /Users/brusilva/venv-grpc/bin/python Installing setuptools, pip, wheel... done.

# activate
$ source venv-grpc/bin/activate

# install grpcio
(venv-grpc)$ pip install grpcio Collecting grpcio Downloading https://files.pythonhosted.org/packages/df/f1/98449d2c173c6324220ab1672203ad09ac7345f023dc62eb0786ad2a0df6/grpcio-1.26.0-cp37-cp37m-macosx_10_9_x86_64.whl (2.3MB) |████████████████████████████████| 2.3MB 1.7MB/s Collecting six>=1.5.2 Downloading https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl Installing collected packages: six, grpcio Successfully installed grpcio-1.26.0 six-1.14.0

# install grpcio-tools
(venv-grpc)$ pip install grpcio-tools Collecting grpcio-tools Downloading https://files.pythonhosted.org/packages/bc/31/5eaf98bec3330004be0700bc9656321feecdf7bd7acf7d25add553176296/grpcio_tools-1.26.0-cp37-cp37m-macosx_10_9_x86_64.whl (1.9MB) |████████████████████████████████| 1.9MB 893kB/s Collecting protobuf>=3.5.0.post1 Downloading https://files.pythonhosted.org/packages/2d/09/176fcab8aab35065e9f973c0ef093e8e296fb4d8d4e3ef2ed4fd2e6ff2f2/protobuf-3.11.2-cp37-cp37m-macosx_10_9_x86_64.whl (1.3MB) |████████████████████████████████| 1.3MB 2.3MB/s Requirement already satisfied: grpcio>=1.26.0 in ./venv-grpc/lib/python3.7/site-packages (from grpcio-tools) (1.26.0) Requirement already satisfied: setuptools in ./venv-grpc/lib/python3.7/site-packages (from protobuf>=3.5.0.post1->grpcio-tools) (45.1.0) Requirement already satisfied: six>=1.9 in ./venv-grpc/lib/python3.7/site-packages (from protobuf>=3.5.0.post1->grpcio-tools) (1.14.0) Installing collected packages: protobuf, grpcio-tools Successfully installed grpcio-tools-1.26.0 protobuf-3.11.2
# note if I try to compile at first it will fail because the gnmi_ext file is in another folder and referenced by url.
# I just copied the gnmi_ext content to the gnmi folder and imported the filename only.

# clone gnmi proto repository

(venv-grpc)$ git clone https://github.com/openconfig/gnmi
Cloning into 'gnmi'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 989 (delta 4), reused 18 (delta 1), pack-reused 961
Receiving objects: 100% (989/989), 492.49 KiB | 1.44 MiB/s, done.
Resolving deltas: 100% (455/455), done.

# failed attempt with original proto
(venv-grpc)$ python -m grpc_tools.protoc -I . --python_out=. gnmi/gnmi.proto github.com/openconfig/gnmi/proto/gnmi_ext/gnmi_ext.proto: File not found. gnmi/gnmi.proto:20:1: Import "github.com/openconfig/gnmi/proto/gnmi_ext/gnmi_ext.proto" was not found or had errors. gnmi/gnmi.proto:214:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:241:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:344:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:363:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:411:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:423:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:432:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:444:12: "gnmi_ext.Extension" is not defined.
# successful compilation after import edit
(venv-grpc)$ python -m grpc_tools.protoc -I . --python_out=. ./gnmi.proto (venv-grpc)$ ls -l total 464 -rw-r--r-- 1 brusilva staff 89684 Jan 19 13:07 gnmi.pb.go -rw-r--r-- 1 brusilva staff 21803 Jan 19 13:11 gnmi.proto -rw-r--r-- 1 brusilva staff 12708 Jan 19 13:12 gnmi_ext.pb.go -rw-r--r-- 1 brusilva staff 2624 Jan 19 13:12 gnmi_ext.proto -rw-r--r-- 1 brusilva staff 10172 Jan 19 13:12 gnmi_ext_pb2.py -rw-r--r-- 1 brusilva staff 83 Jan 19 13:12 gnmi_ext_pb2_grpc.py -rw-r--r-- 1 brusilva staff 76035 Jan 19 13:14 gnmi_pb2.py
-rw-r--r-- 1 brusilva staff 4864 Jan 19 13:07 gnmi_pb2_g

There you go! Now you see these python files created. They are ready to be used now.

Now all you need is to import them into your python code.

(venv-grpc)$ python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from gnmi.gnmi_pb2_grpc import gNMIStub
>>> from gnmi.gnmi_pb2 import GetRequest, GetResponse, Path, PathElem, CapabilityRequest, Encoding, SetRequest, Update, TypedValue

Router configuration

Following is the basic gRPC config needed:

grpc
 port 57400
 address-family dual <<< to enable for ipv4 and ipv6
 max-request-total 256
 max-request-per-user 32

Coding

Certificate and credentials

As part of the authentication process, you will need the pem file (certificate) from the router(s) you want to communicate with. Download the following file from the router(s): /misc/config/grpc/ems.pem.

 

Variable-wise, we will use the following bare bones:

with open("ems.pem", "rb") as fp: <<< make sure to read the pem as bytes string
    pem = fp.read()
host = "10.8.70.51"
port = "57400" <<< this is the port defined in the gRPC config on the router
metadata=[('username', "username"), ('password', "password")]
options=[('grpc.ssl_target_name_override', 'ems.cisco.com'), ('grpc.max_receive_message_length', 1000000000)]

The options variable defines what url to be used for the ssl verification, which for Cisco should be ems.cisco.com, and that is needed for any IOS-XR router.

The message_length we are changing from the 4MB default.

You can check all the options available in this url: https://grpc.github.io/grpc/core/group__grpc__arg__keys.html

The rest is self-explanatory. 

Sending a GetRequest

Before we actually establish the connection, let's first read the proto file to understand what we need to do in case we want to send a GetRequest for example.

https://github.com/openconfig/gnmi/blob/master/proto/gnmi/gnmi.proto
// GetRequest is sent when a client initiates a Get RPC. It is used to specify // the set of data elements for which the target should return a snapshot of // data. The use_models field specifies the set of schema modules that are to // be used by the target - where use_models is not specified then the target // must use all schema models that it has. // Reference: gNMI Specification Section 3.3.1 message GetRequest { Path prefix = 1; // Prefix used for paths. repeated Path path = 2; // Paths requested by the client. // Type of elements within the data tree. enum DataType { ALL = 0; // All data elements. CONFIG = 1; // Config (rw) only elements. STATE = 2; // State (ro) only elements. // Data elements marked in the schema as operational. This refers to data // elements whose value relates to the state of processes or interactions // running on the device. OPERATIONAL = 3; } DataType type = 3; // The type of data being requested. Encoding encoding = 5; // Encoding to be used. repeated ModelData use_models = 6; // The schema models to be used. // Extension messages associated with the GetRequest. See the // gNMI extension specification for further definition. repeated gnmi_ext.Extension extension = 7; }

As you can see, in order to send a request, we basically need:

 

1. prefix of type Path - basically what we are polling.

2. a list of path items of type Path, which is what we are requesting.

3. type of type DataType. This is what type of data we want. We have 4 options basically.

4. encoding of type Encoding. This specifies how we will serialize the data over the gRPC connection. As of this article, we support JSON_IETF and ASCII format.

 

So in every GetRequest, I have to specify the above as a minimum.

 

So 1st step is to actually create a credential with the pem file:

credentials = grpc.ssl_channel_credentials(pem) # object type is grpc.ChannelCredentials

Now we are ready to establish the connection.

    try:
        channel = grpc.secure_channel(':'.join([host, port]), credentials, options) <<< create the secure channel
        grpc.channel_ready_future(channel).result(timeout=10) <<< we specify the timeout as 10s and provide the channel
        gnmi_stub = gNMIStub(channel) <<< creating the stub object and providing the channel
        get_path = create_gnmi_path("Cisco-IOS-XR-ip-tcp-cfg:ip-tcp") <<< in this example we are polling the ip-tcp config
        get_message = GetRequest(path=[get_path], type=1, encoding=4) <<< paths are passed as a List. type 1 = CONFIG, encoding 4 is JSON_IETF
        print(gnmi_stub.Get(get_message, metadata=metadata)) <<< printing the raw response
    except grpc.FutureTimeoutError as e: <<< always check for TimeoutError
        print(e)
        print("Failed to connect")

Note that this create_gnmi_path function we created to just convert the model specified in green into a request with the following format:

path {
  elem {
    name: "Cisco-IOS-XR-ip-tcp-cfg:ip-tcp"
  }
}
encoding: JSON_IETF

Note that while this Path looks like json, it has to be formatted and created using the classes based on the proto file. Here's how a Path message looks like:

message Path {
  // Elements of the path are no longer encoded as a string, but rather within
  // the elem field as a PathElem message.
  repeated string element = 1 [deprecated=true];
  string origin = 2;                              // Label to disambiguate path.
  repeated PathElem elem = 3;                     // Elements of the path.
  string target = 4;                              // The name of the target
                                                  // (Sec. 2.2.2.1)
}

origin and target are not necessary in this case.

What we need is the list of PathElem objects elem. In our case we are only using one path, but know you can do multiple.

Each PathElem is defined as follows:

message PathElem {
  string name = 1;                    // The name of the element in the path.
  map<string, string> key = 2;        // Map of key (attribute) name to value.
}

so you can create one PathElem like this for example, and then put that into a list, and lastly you can create the Path object:

# create elem
elem_name = 'Cisco-IOS-XR-ip-tcp-cfg:ip-tcp' path_elem = PathElem(name=elem_name, key={})

# append elem to a list as defined in proto file
list_of_path_elems = []
list_of_path_elems.append(path_elem)

# create Path object
p = Path(elem=list_of_path_elems)

So then when you send a Get, you will send "p" object.

 

This is how we get the response:

notification { <<< every response in gNMI is returned in a notification protocol buffer
  timestamp: 1575572561137134733 <<< always timestamped. nanoseconds since epoch.
  update { <<< update is a list of items
    path {
      elem {
        name: "Cisco-IOS-XR-ip-tcp-cfg:ip-tcp" <<< name is the model we are polling.
      }
    }
    val { <<< within val is our response
      json_ietf_val: "{\"path-mtu-discovery\":10,\"syn-wait-time\":5}" <<< encoding type is json_ietf, so we get it in that format
    }
  }
}
error {
}

So you can compare and get used to the proto buff language definition syntax, here's how the notification message looks like from the gnmi proto file below.

Just like the response we get, we have timestamp and prefix (path). And the update is defined as a "repeated" (or "list" in python).

 

message Notification {
  int64 timestamp = 1;          // Timestamp in nanoseconds since Epoch.
  Path prefix = 2;              // Prefix used for paths in the message.
  // An alias for the path specified in the prefix field.
  // Reference: gNMI Specification Section 2.4.2
  string alias = 3;
  repeated Update update = 4;   // Data elements that have changed values.
  repeated Path delete = 5;     // Data elements that have been deleted.
  // This notification contains a set of paths that are always updated together
  // referenced by a globally unique prefix.
  bool atomic = 6;
}

Note we also have the response itself that comes as a val key. Look at the proto file for the update message type:

 

message Update {
  Path path = 1;                      // The path (key) for the update.
  Value value = 2 [deprecated=true];  // The value (value) for the update.
  TypedValue val = 3;                 // The explicitly typed update value.
  uint32 duplicates = 4;              // Number of coalesced duplicates.
}

The encoding is of type json_ietf (we support only that and ascii), and the response comes as bytes string that you can convert to dictionary. See below how the proto file defines the TypedValue message type:

 

message TypedValue {
  // One of the fields within the val oneof is populated with the value
  // of the update. The type of the value being included in the Update
  // determines which field should be populated. In the case that the
  // encoding is a particular form of the base protobuf type, a specific
  // field is used to store the value (e.g., json_val).
  oneof value {
    string string_val = 1;            // String value.
    int64 int_val = 2;                // Integer value.
    uint64 uint_val = 3;              // Unsigned integer value.
    bool bool_val = 4;                // Bool value.
    bytes bytes_val = 5;              // Arbitrary byte sequence value.
    float float_val = 6;              // Floating point value.
    Decimal64 decimal_val = 7;        // Decimal64 encoded value.
    ScalarArray leaflist_val = 8;     // Mixed type scalar array value.
    google.protobuf.Any any_val = 9;  // protobuf.Any encoded bytes.
    bytes json_val = 10;              // JSON-encoded text.
    bytes json_ietf_val = 11;         // JSON-encoded text per RFC7951. <<< this is the type we are getting    string ascii_val = 12;            // Arbitrary ASCII text.
    // Protobuf binary encoded bytes. The message type is not included.
    // See the specification at
    // github.com/openconfig/reference/blob/master/rpc/gnmi/protobuf-vals.md
    // for a complete specification.
    bytes proto_bytes = 13;
  }
}

Polling Operational Data

When you want to use gnmi for get-oper, you need not only the yang model but also the top-level containers. If you run a GetCapability to the router, you only get the models but not the top-level containers.

For production scenarios, ideally you will have a list of models + paths you want to pull, and then just run a get for each.

You can also specify all models at once. In gRPC terms each model is a 

You should then be able to loop through everything you defined if you wanted to get all oper models in a file for example.

 

        for model in models: <<< models would be a list of paths
            get_path = create_gnmi_path(model)
            get_message = GetRequest(path=[get_path], type=3, encoding=4) <<< type 3 = OPERATIONAL
            try:
                out = gnmi_stub.Get(get_message, metadata=metadata)
            except Exception as e:
                print(e)
                print(model)

This is just a simple example. Do it however you like

Polling Configuration Data

Partial Configuration

In order to pull config data via gnmi, you have the option to specify the model and top-level containers for the model in question - that's 1 way of doing it.

Here's an example, note this is just a snippet of the actual code:

models = ['Cisco-IOS-XR-ipv4-bgp-cfg:bgp'] <<< specify model:top-level-container in a list
    for model in models:
        get_path = create_gnmi_path(model)
        get_message = GetRequest(path=[get_path],type=GetRequest.DataType.Value("CONFIG"), encoding=Encoding.Value("JSON_IETF"))
        try:
            out = gnmi_stub.Get(get_message, metadata=metadata)
            print(f'Successfully got {model}\n\n{out}')
        except Exception as e:
            print(f'Error polling {model}. Exception:\n{e}')

And the response:

notification {
  timestamp: 1575598106554876261
  update {
    path {
      elem {
        name: "Cisco-IOS-XR-ipv4-bgp-cfg:bgp"
      }
    }
    val {
      json_ietf_val: "{\"instance\":[{\"instance-name\":\"default\",\"instance-as\":[{\"as\":0,\"four-byte-as\":[{\"as\":100,\"bgp-running\":[null],\"default-vrf\":{\"global\":{\"router-id\":\"1.1.1.5\",\"graceful-restart-time\":120,\"graceful-restart-stalepath-time\":360,\"graceful-restart\":[null],\"neighbor-logging-detail\":[null],\"best-path-med-always\":[null],\"best-path-router-id\":[null],\"enforce-ibgp-out-policy\":[null],\"global-afs\":{\"global-af\":[{\"af-name\":\"ipv4-unicast\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\",\"attribute-download\":[null],\"ebgp\":{\"paths-value\":32,\"unequal-cost\":false,\"selective\":false,\"order-by-igp-metric\":false},\"ibgp\":{\"paths-value\":32,\"unequal-cost\":false,\"selective\":false,\"order-by-igp-metric\":false}},{\"af-name\":\"vpnv4-unicast\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\"},{\"af-name\":\"ipv6-unicast\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\",\"ebgp\":{\"paths-value\":32,\"unequal-cost\":false,\"selective\":false,\"order-by-igp-metric\":false},\"ibgp\":{\"paths-value\":32,\"unequal-cost\":false,\"selective\":false,\"order-by-igp-metric\":false}},{\"af-name\":\"vpnv6-unicast\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\"},{\"af-name\":\"l2vpn-vpls\",\"enable\":[null]},{\"af-name\":\"ipv4-mvpn\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\"}]}},\"bgp-entity\":{\"neighbor-groups\":{\"neighbor-group\":[{\"neighbor-group-name\":\"IBGP-RR\",\"create\":[null],\"remote-as\":{\"as-xx\":0,\"as-yy\":100},\"update-source-interface\":\"Loopback0\",\"neighbor-group-afs\":{\"neighbor-group-af\":[{\"af-name\":\"ipv4-unicast\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"vpnv4-unicast\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"ipv6-unicast\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"vpnv6-unicast\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"l2vpn-vpls\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"ipv4-mvpn\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}}]}}]},\"neighbors\":{\"neighbor\":[{\"neighbor-address\":\"1.1.1.1\",\"neighbor-group-add-member\":\"IBGP-RR\",\"description\":\"RR-II11-5501-Bran\"},{\"neighbor-address\":\"1.1.1.6\",\"neighbor-group-add-member\":\"IBGP-RR\",\"description\":\"RR-II10-XRv9k-Varys\"}]}}},\"vrfs\":{\"vrf\":[{\"vrf-name\":\"spoke2\",\"vrf-global\":{\"exists\":[null],\"route-distinguisher\":{\"type\":\"as\",\"as-xx\":0,\"as\":200,\"as-index\":200},\"vrf-global-afs\":{\"vrf-global-af\":[{\"af-name\":\"ipv4-unicast\",\"enable\":[null],\"connected-routes\":{\"default-metric\":1}}]}},\"vrf-neighbors\":{\"vrf-neighbor\":[{\"neighbor-address\":\"5.20.0.2\",\"remote-as\":{\"as-xx\":0,\"as-yy\":65002},\"vrf-neighbor-afs\":{\"vrf-neighbor-af\":[{\"af-name\":\"ipv4-unicast\",\"activate\":[null],\"route-policy-in\":\"pass\",\"route-policy-out\":\"pass\"}]}}]}},{\"vrf-name\":\"mvpn_p10_vrf1\",\"vrf-global\":{\"exists\":[null],\"route-distinguisher\":{\"type\":\"as\",\"as-xx\":0,\"as\":10,\"as-index\":9},\"vrf-global-afs\":{\"vrf-global-af\":[{\"af-name\":\"ipv4-unicast\",\"enable\":[null],\"connected-routes\":{}},{\"af-name\":\"ipv4-mvpn\",\"enable\":[null]}]}}}]}}]}]}]}"
    }
  }
}
error {
}

Same way as polling a operational model, the response itself is inside the json_ietf_val key.

Full Configuration

What if you want to pull the full-config? Very easy! Just specify an empty Path object. Like this:

get_path = create_gnmi_path(model)
get_message = GetRequest(path=[Path()], type=GetRequest.DataType.Value("CONFIG"), encoding=Encoding.Value("JSON_IETF"))
try:
    out = gnmi_stub.Get(get_message, metadata=metadata)
except Exception as e:
    print(f'Error during get-config. Exception:\n{e}')

Then just use out.full_config to see the config. it is returned as a json/dict object already.

Changing Configuration

We use a SetRequest message type to change any configuration via gNMI.

// SetRequest is sent from a client to the target to update values in the data
// tree. Paths are either deleted by the client, or modified by means of being
// updated, or replaced. Where a replace is used, unspecified values are
// considered to be replaced, whereas when update is used the changes are
// considered to be incremental. The set of changes that are specified within
// a single SetRequest are considered to be a transaction.
// Reference: gNMI Specification Section 3.4.1
message SetRequest {
  Path prefix = 1;                // Prefix used for paths in the message.
  repeated Path delete = 2;       // Paths to be deleted from the data tree.
  repeated Update replace = 3;    // Updates specifying elements to be replaced.
  repeated Update update = 4;     // Updates specifying elements to updated.
  // Extension messages associated with the SetRequest. See the
  // gNMI extension specification for further definition.
  repeated gnmi_ext.Extension extension = 5;
}

So we need the following information to change a config:

 

1. prefix = that is the path, or model we want to touch. It is a Path object.

2. update or replace = that is the list of updates or replacements we have - this is where your config will actually be stored. It is a list because you can change several different configs in a single transaction. It is a Update object.

 

In order to create an Update object, let's review the message format.

// Update is a re-usable message that is used to store a particular Path,
// Value pair.
// Reference: gNMI Specification Section 2.1
message Update {
  Path path = 1;                      // The path (key) for the update.
  Value value = 2 [deprecated=true];  // The value (value) for the update.
  TypedValue val = 3;                 // The explicitly typed update value.
  uint32 duplicates = 4;              // Number of coalesced duplicates.
}

In here we need:

 

1. A Path object called path, which contains the model we want to change.

2. a TypedValue object val which has the config we want to replace or update.

 

Now that we know what we need, take a look at the config example below and sample code:

{
	"interface-configuration": [{
		"active": "act",
		"interface-name": "Loopback10",
		"interface-virtual": [
			null
		],
		"Cisco-IOS-XR-ipv4-io-cfg:ipv4-network": {
			"addresses": {
				"primary": {
					"address": "1.1.1.12",
					"netmask": "255.255.255.255"
				}
			}
		},
		"Cisco-IOS-XR-ipv6-ma-cfg:ipv6-network": {
			"addresses": {
				"regular-addresses": {
					"regular-address": [{
						"address": "2001:1:1:1::12",
						"prefix-length": 128,
						"zone": "0"
					}]
				}
			}
		}
	}]
}

The easiest way to get that config is to pull (get-config), make some modifications as you'd like, and push it (set). In this case, I will add a new loopback10 interface.

Note that the top-level model is not specified here. this will be in the Update object we create.

 

See the code below now:

# first we open the config file        
config_json: dict = self.read_config(config_file) # define the path string, which is the model we are targeting
path: str = "Cisco-IOS-XR-ifmgr-cfg:interface-configurations" # then create Path object with path from filename
path_object: Path = self._create_gnmi_path(path) # The config in json is then used to create the TypedValue object in json_ietf_val format.
type_value: TypedValue = TypedValue(json_ietf_val=config_json) # Create Update message
update_object: Update = Update(path=path_object, val=type_value) # Here we do an update or replace based on the request.
if replace_or_update == "replace": set_request_object: SetRequest = SetRequest(replace=[update_object]) else: set_request_object: SetRequest = SetRequest(update=[update_object]) print(f'set_request_object: {set_request_object}') # Create the gnmi_stub to open the channel and send the Set. Store response in "response" which is type SetResponse self.gnmi_stub: gNMIStub = gNMIStub(self.channel) response: SetResponse = self.gnmi_stub.Set(set_request_object, metadata=self.metadata)

Subscription

Another feature of gRPC is the ability to subscribe to a certain Path and specify a desired interval. This is a great feature to allow the client to define what it wants from the router, without having to configure anything on it - all the information is in the application!

I am going to use the same bare bones script as above, just modifying to what we need - same variable names and all.

 

So while I show you how to code this, let's also learn the new message types we are going to handle.

I will handle this backwards, as we have multiple different message types and I think it's easier to go that way to understand first.

So the RPC itself is Subscribe, which gets a SubscribeRequest and returns a SubscribeResponse:

 

rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse);

The way I am processing the SubscribeResponse's is by creating a generator to walk over the "models" i am interested in. Like so:

def subscribe_to_path(request):
    yield request

# stub is already created here for response in gnmi_stub.Subscribe(subscribe_to_path(subscribe_request), metadata=gnmi_metadata): print(response)

Let's look at the SubscribeRequest message format:

message SubscribeRequest {
  oneof request {
    SubscriptionList subscribe = 1; // Specify the paths within a subscription.
    Poll poll = 3;                  // Trigger a polled update.
    AliasList aliases = 4;          // Aliases to be created.
  }
  // Extension messages associated with the SubscribeRequest. See the
  // gNMI extension specification for further definition.
  repeated gnmi_ext.Extension extension = 5;
}

So in our case, we will looking at a SubscriptionList. You can also specify a polled update.

A SubscribeRequest asks for a SubscriptionList. That means all models you specify in a SubscriptionList are considered to be a part of one single Subscription. Let's create then a SubscribeRequest:

 

# we provide the subscriptionlist to create a SubscribeRequest message
subscribe_request = SubscribeRequest(subscribe=subscriptionlist)

So now we need to create the SubscriptionList, which in my case is subscriptionlist. Below is the message format:

message SubscriptionList {
  Path prefix = 1;                          // Prefix used for paths.
  repeated Subscription subscription = 2;   // Set of subscriptions to create.
  // Whether target defined aliases are allowed within the subscription.
  bool use_aliases = 3;
  QOSMarking qos = 4;                       // DSCP marking to be used.
  // Mode of the subscription.
  enum Mode {
    STREAM = 0; // Values streamed by the target (Sec. 3.5.1.5.2).
    ONCE = 1;   // Values sent once-off by the target (Sec. 3.5.1.5.1).
    POLL = 2;   // Values sent in response to a poll request (Sec. 3.5.1.5.3).
  }
  Mode mode = 5;
  // Whether elements of the schema that are marked as eligible for aggregation
  // should be aggregated or not.
  bool allow_aggregation = 6;
  // The set of schemas that define the elements of the data tree that should
  // be sent by the target.
  repeated ModelData use_models = 7;
  // The encoding that the target should use within the Notifications generated
  // corresponding to the SubscriptionList.
  Encoding encoding = 8;
  // An optional field to specify that only updates to current state should be
  // sent to a client. If set, the initial state is not sent to the client but
  // rather only the sync message followed by any subsequent updates to the
  // current state. For ONCE and POLL modes, this causes the server to send only
  // the sync message (Sec. 3.5.2.3).
  bool updates_only = 9;
}

The path will be part of the Subscription - we will look at that next.

The Subscription messages are defined in a list, in order to allow us to specify multiple models in the same Subscription. 

subscriptionlist_mode = SubscriptionList.Mode.Value("STREAM")
subscriptionlist_encoding = Encoding.Value("PROTO")
subscriptionlist = SubscriptionList(subscription=subscriptions,
mode=subscriptionlist_mode,
encoding=subscriptionlist_encoding)

Note the mode and encoding. We are using STREAM mode, with PROTO encoding. You are welcome to use JSON as well, but PROTO is leaner especially with high amounts of data.

You can use the following modes:

  enum Mode {
    STREAM = 0; // Values streamed by the target (Sec. 3.5.1.5.2).
    ONCE = 1;   // Values sent once-off by the target (Sec. 3.5.1.5.1).
    POLL = 2;   // Values sent in response to a poll request (Sec. 3.5.1.5.3).
  }

Now onto the last message type we need to know - Subscription:

message Subscription {
  Path path = 1;                    // The data tree path.
  SubscriptionMode mode = 2;        // Subscription mode to be used.
  uint64 sample_interval = 3;       // ns between samples in SAMPLE mode.
  // Indicates whether values that have not changed should be sent in a SAMPLE
  // subscription.
  bool suppress_redundant = 4;
  // Specifies the maximum allowable silent period in nanoseconds when
  // suppress_redundant is in use. The target should send a value at least once
  // in the period specified.
  uint64 heartbeat_interval = 5;
}
Path we already know how to build, and mode can be one of the following:
enum SubscriptionMode {
  TARGET_DEFINED = 0;  // The target selects the relevant mode for each element.
  ON_CHANGE      = 1;  // The target sends an update on element value change.
  SAMPLE         = 2;  // The target samples values according to the interval.
}
You can leave it up to the server to define the type of subscription (TARGED_DEFINED), use ON_CHANGE (i.e. for administrative status of an interface), or SAMPLE, which is always going to dump the values in an interval specified.

 

Also note here that the sample_interval is in nanoseconds!  Make sure to not try to subscribe to a huge model every 10ns

Now onto how we actually instantiate this message. In my case I am using two Subscription messages. That means 1 SubscribeRequest will ask for data about two paths specified in "models".

# Note we are doing what's called leaf-level streaming. This is supported on 721 onwards
models = ["openconfig-interfaces:interfaces/interface[name=TenGigE0/0/0/18/0]/state/counters/out-octets", "openconfig-interfaces:interfaces/interface[name=TenGigE0/0/0/18/1]/state/counters/out-octets"] # I personally like to specify the interval always in seconds and multiply after
sample_interval = 30 sample_rate = sample_interval*1000000000 # subscribe mode is just a string
subscribe_mode = "SAMPLE" subscriptions = [] for model in models:
# create the Path object sub_path = create_gnmi_path(model)
# create the SubscriptionMode object sub_mode = SubscriptionMode.Value(subscribe_mode)
# create the Subscription sub_subscription = Subscription(path=sub_path, mode=sub_mode, sample_interval=sample_rate) subscriptions.append(sub_subscription)

Now we got all the pieces! Remember the chain looks like this:

 

Subscription --> SubscriptionList --> SubscribeRequest --> Subscribe

SAMPLE

Let's actually run it on a router in SAMPLE mode and see what we get. I put the time below so you can see what happens.

-- 20:06:53 --
# This is the first response from the router - the 1st model
update {
  timestamp: 1588464413328000000
  prefix {
    origin: "openconfig-interfaces"
    elem {
      name: "interfaces"
    }
    elem {
      name: "interface"
      key {
        key: "name"
        value: "TenGigE0/0/0/18/0"
      }
    }
    elem {
      name: "state"
    }
    elem {
      name: "counters"
    }
  }
  update {
    path {
      elem {
        name: "out-octets"
      }
    }
    val {
      uint_val: 226343920640
    }
  }
}

-- 20:06:53 --
# This is the 2nd response - to the 2nd model
update {
  timestamp: 1588464413708000000
  prefix {
    origin: "openconfig-interfaces"
    elem {
      name: "interfaces"
    }
    elem {
      name: "interface"
      key {
        key: "name"
        value: "TenGigE0/0/0/18/1"
      }
    }
    elem {
      name: "state"
    }
    elem {
      name: "counters"
    }
  }
  update {
    path {
      elem {
        name: "out-octets"
      }
    }
    val {
      uint_val: 243752831062
    }
  }
}

-- 20:06:55 --
# This is a sync message after the subscriptions were all received
sync_response: true

What you notice here is. The update dictionary contains an update dictionary with the actual response of the out-octets from the router. It contains the path element name, as well as the value in unsigned integer as expected.

 

You will see now that the session is always going to be up, and every <interval>, you will get a new "SAMPLE" of the models you requested. Pretty cool, right?!

All you need to do after this is format the data as you wish for the database of your preference and start graphing it out!

ON_CHANGE

Now say you want to just get updates for a leaf or container when there are changes. You don't really need to poll at a certain interval - just keep the subscription established and when you subscribe, use the ON_CHANGE mode instead of SAMPLE.

Your subscription message would look like this for the following model:

 

# openconfig-interfaces:interfaces/interface[name=TenGigE0/0/0/18/0]/state/admin-status
subscription { path { elem { name: "openconfig-interfaces:interfaces" } elem { name: "interface" key { key: "name" value: "TenGigE0/0/0/18/0" } } elem { name: "state" } elem { name: "admin-status" } } mode: ON_CHANGE sample_interval: 10000000000 } encoding: PROTO

Immediately when the admin-status of this interface changes to any of the states defined in the model, you will receive a new update! That's the best way to deal with somewhat static data!

 

One important note - you need to use the following cli if you wish to receive update only about the leaf you are subscribing to, vs the whole bag or gathering point for the model.

telemetry model-driven
 include select-leaves-on-events

This is by default how TARGET_DEFINED works, which is described below.

TARGET_DEFINED

Let's say you don't know the best option, whether ON_CHANGE or SAMPLE for a particular leaf. You can use TARGET_DEFINED to allow the router to decide how to send the data, whether at SAMPLE mode or ON_CHANGE mode. Your Subscription would look like below - note that the mode is unspecified.

 

subscription {
  path {
    elem {
      name: "openconfig-interfaces:interfaces"
    }
    elem {
      name: "interface"
      key {
        key: "name"
        value: "TenGigE0/0/0/18/0"
      }
    }
    elem {
      name: "state"
    }
    elem {
      name: "admin-status"
    }
  }
  sample_interval: 10000000000
}

API

In order to simplify the usage of GNMI using python scripts, I've created a simple library for GNMI Subscriptions:

https://github.com/brunoonovais/GNMIStub

Comments
fracjackmac
Level 1
Level 1

Thanks for the excellent post @Bruno De Oliveira Novais!

 

@StuartClark (@bigevilbeard) posted a link to this article on Twitter ...

 

 

Thanks again!!

 

 

@ittybittypacket

fjm

taoli
Cisco Employee
Cisco Employee

不错, 谢谢分享, 学习了。Thanks Sharing.

douglasfir
Level 1
Level 1

Hi Bruno,

 

thanks a lot for sharing this article. Do you maybe have complete source code of this example? It would mean a lot to me, as I was trying to run it, but I run onto many obstacles, it could be that I didn't understand each step completely.

I was following all the steps until part where connection is being established and "get" request is sent. I adapted code a bit and tried to run, but I get the following error:

 

TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "gnmi/gnmi.proto":
  gnmi_ext/gnmi_ext.proto: Import "gnmi_ext/gnmi_ext.proto" has not been loaded.
  gnmi.SubscribeRequest.extension: "gnmi_ext.Extension" seems to be defined in "proto/gnmi_ext/gnmi_ext.proto", which is not imported by "gnmi/gnmi.proto".  To use it here, please add the necessary import.
  gnmi.SubscribeResponse.extension: "gnmi_ext.Extension" seems to be defined in "proto/gnmi_ext/gnmi_ext.proto", which is not imported by "gnmi/gnmi.proto".  To use it here, please add the necessary import.
  gnmi.SetRequest.extension: "gnmi_ext.Extension" seems to be defined in "proto/gnmi_ext/gnmi_ext.proto", which is not imported by "gnmi/gnmi.proto".  To use it here, please add the necessary import.
.......

I was using the following script:

import grpc
from jinja2 import Template
from gnmi.gnmi_pb2_grpc import gNMIStub
from gnmi.gnmi_pb2 import GetRequest, GetResponse, Path, PathElem, \
    CapabilityRequest, Encoding, SetRequest, Update, TypedValue

host = "198.18.1.11"
port = "57400"
metadata = [('username', "admin"), ('password', "admin")]
options = [('grpc.ssl_target_name_override', 'ems.cisco.com'), ('grpc.max_receive_message_length', 1000000000)]


def create_gnmi_path(path):
    request_jinja = '''
    path {
  elem {
    name: {{path}}
  }
}
encoding: JSON_IETF'''
    return Template(request_jinja).render(path=path)

try:
    channel = grpc.insecure_channel(':'.join([host, port]), options)
    grpc.channel_ready_future(channel).result(timeout=10)
    gnmi_stub = gNMIStub(channel)
    get_path = create_gnmi_path("Cisco-IOS-XR-ip-tcp-cfg:ip-tcp")
    get_message = GetRequest(path=[get_path], type=1, encoding=4)
    print(gnmi_stub.Get(get_message, metadata=metadata))
except grpc.FutureTimeoutError as e:
    print(e)
    print("Failed to connect")

Thanks in advance!

Dragan

Bruno De Oliveira Novais
Cisco Employee
Cisco Employee

Hey@douglasfir !

So i've updated the article to include the details of how you create the Path object - that's the part you needed, so that should help you finalize your basic code :)

 

Cheers!

Bruno Novais

douglasfir
Level 1
Level 1

Hi Bruno,

thanks a lot for the update!

BR, Dragan

Hi Bruno

 

Thanks for the great blog post.

 

Can you tell me if all the gnmi subscribe "modes" below are supported on IOS-XR ? ,

 

  TARGET_DEFINED = 0;  // The target selects the relevant mode for each element.
  ON_CHANGE      = 1;  // The target sends an update on element value change.
  SAMPLE         = 2;  // The target samples values according to the interval.
}

 

 

I have recently tried Cisco IOS XE Software, Version 16.12.03a  in 

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/prog/configuration/172/b_172_programmability_cg/model_driven_telemetry.html#id_90796   

 

it states “In Cisco IOS XE Gibraltar 16.12.1, only SAMPLE is supported" - are there plans to support all the subscribe modes in IOS XE ? 

 

Or in which Cisco IOSes are all the modes supported ?

 

With regards to the subscribe functionality and 16.12.03a 

 

I noticed that for subscriptions, the path needs to be fully qualified , and contain no wildcards -> e.g “subscribe interfaces/interface[name=*]/ethernet/state/port-speed” , will fail , but a get on the same path will work ok.

 

Do you know if wildcard subscriptions will work in other versions of the SW - eg IOS-XR ?

 

many thanks

 

Jamie

 

 

 

 

 

 

Bruno De Oliveira Novais
Cisco Employee
Cisco Employee

Hi Jamie

 

As of release 71x on IOS-XR we should support all modes.

And we don't (yet) support wildcard - ideally you provide all keys in subscription you desire. i.e.: openconfig-interfaces:interfaces/interface[name=TenGigE0/0/0/18/0]/state.

 

I don't have the latest info on the ios/ios-xe support though, but i am sure tac could help you get that info easily!

 

Cheers!

Bruno Novais

Hi Bruno

 

Many thanks for the information.

 

Would it possible for someone to provide a timeline when the 3 subscription modes would be supported on the other ioses ?

 

And again a timeline for when subscribe with wildcards would be supported on all the ioses ?

 

best regards

 

Jamie

 

 

 

asilverstein
Level 1
Level 1

Hello Bruno,

Thx for the great article.  I'm trying to do the same exact thing for different vendor equipment.  Installation seems to have gone well but I can't seem to get the script to even initiate a connection to my device.  It gets hung up on this:

grpc.channel_ready_future(channel).result(timeout=10)

It timeouts and when I have tcpdump running on the Linux interface, a connection is never attempted. Any ideas how to troubleshoot this?  I'm looking for logs or debugging info but I can't seem to find anything.

thx

Al

Bruno De Oliveira Novais
Cisco Employee
Cisco Employee

Hi @asilverstein ! Thx!

I believe if you enable debug mode on python it should give you some more info. Another useful general python technique i use is using python interactive mode (code.interact())...that should give you a better idea of what's going on.

 

Thx

Bruno Novais

asilverstein
Level 1
Level 1

I tried doing this but nothing is written the log file..  I'm not familiar with code.interact() but I'll look into it

 

import logging
logging.basicConfig(filemode="w", filename="gnmi_app.log",
format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s',
datefmt='%H:%M:%S',level=logging.DEBUG,)
logging.getLogger("gnmi_app")

 

I'm not familiar with code.interact() but I'll look into it

Thx

Al

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Quick Links