- Introduction
- gRPC Terminology and Installation
- Router configuration
- Coding
- Certificate and credentials
- Sending a GetRequest
- Polling Operational Data
- Polling Configuration Data
- Partial Configuration
- Full Configuration
- Changing Configuration
- Subscription
- SAMPLE
- ON_CHANGE
- TARGET_DEFINED
- API
Introduction
While gNMI is fairly new, it's becoming more and more powerful. Its abilities to simplify network management by the use of protocol buffer files and standard definitions are enabling our customers to integrate a lot better in multi-vendor environments.
It's also becoming more common to see aspiring developers (aka network engineers who know some coding) dealing with responsibilities of automating networks previously managed either manually or via old-fashioned scripts. It may seem a bit difficult to understand gNMI at first for a network engineer (or not if you're too smart!).
In this document I will go over the basics of setting up the stage for gNMI, as well as showing practical and working code as an example. Hopefully it simplifies the lives of these engineers, as well as showing the power of gNMI on IOS-XR.
This is not intended to show you how to build scaled software to support thousands of routers. It is instead going to help you get started. Once the building blocks are done, you can scale this as much as you want with the knowledge you have!
You might ask, why would I want to use gNMI at all when netconf is out there? The main reason is performance. By compressing data in binary format, gRPC can transfer a lot more data than xml, which netconf uses. About 3x - 10x faster. No big deal when you're dealing with 1 router, or maybe even hundreds. But can you manage hundreds of thousands of routers by transferring huge amounts of data all the time from them? Probably not!
Also, say you got some Python engineers, but tomorrow you hire some Go engineers. Guess what...the code is virtually the same between them - the libraries that we compile work the same way whether you use python, go, c++, etc. You don't have to build api's from scratch to manage a device, the proto buffer file is compiled into code for your use right away.
Take a look at the table below as well to get some more comparisons to similar methods:
| Features | netconf | cisco grpc | openconfig gnmi |
| Transport | ssh | http2 | http2 |
| Support | vendor neutral | proprietary | vendor neutral |
| Encoding | xml | proto buff | proto buff |
gRPC Terminology and Installation
In practical terms, the gRPC Stub will be your *nix server. The IOS-XR router will be the gRPC Server. See below image for some clarity:
Image from https://www.grpc.io/docs/guides/
Your proto file contains all the definitions of how the router in this case will receive requests and how to reply. These definitions are defined here: https://github.com/openconfig/gnmi/blob/master/proto/gnmi/gnmi.proto
For Python, all the instructions to install the gRPC tools needed are in the following link: https://www.grpc.io/docs/quickstart/python/
It's self-explanatory, but simple steps are:
1. Install grpcio
2. Install grpc-io tools - that includes the protoc, the compiler of the protocol buffer files.
3. run compiler
# create a virtual environment
$ virtualenv venv-grpc Using base prefix '/Library/Frameworks/Python.framework/Versions/3.7' New python executable in /Users/brusilva/venv-grpc/bin/python3.7 Also creating executable in /Users/brusilva/venv-grpc/bin/python Installing setuptools, pip, wheel... done.
# activate
$ source venv-grpc/bin/activate
# install grpcio
(venv-grpc)$ pip install grpcio Collecting grpcio Downloading https://files.pythonhosted.org/packages/df/f1/98449d2c173c6324220ab1672203ad09ac7345f023dc62eb0786ad2a0df6/grpcio-1.26.0-cp37-cp37m-macosx_10_9_x86_64.whl (2.3MB) |████████████████████████████████| 2.3MB 1.7MB/s Collecting six>=1.5.2 Downloading https://files.pythonhosted.org/packages/65/eb/1f97cb97bfc2390a276969c6fae16075da282f5058082d4cb10c6c5c1dba/six-1.14.0-py2.py3-none-any.whl Installing collected packages: six, grpcio Successfully installed grpcio-1.26.0 six-1.14.0
# install grpcio-tools
(venv-grpc)$ pip install grpcio-tools Collecting grpcio-tools Downloading https://files.pythonhosted.org/packages/bc/31/5eaf98bec3330004be0700bc9656321feecdf7bd7acf7d25add553176296/grpcio_tools-1.26.0-cp37-cp37m-macosx_10_9_x86_64.whl (1.9MB) |████████████████████████████████| 1.9MB 893kB/s Collecting protobuf>=3.5.0.post1 Downloading https://files.pythonhosted.org/packages/2d/09/176fcab8aab35065e9f973c0ef093e8e296fb4d8d4e3ef2ed4fd2e6ff2f2/protobuf-3.11.2-cp37-cp37m-macosx_10_9_x86_64.whl (1.3MB) |████████████████████████████████| 1.3MB 2.3MB/s Requirement already satisfied: grpcio>=1.26.0 in ./venv-grpc/lib/python3.7/site-packages (from grpcio-tools) (1.26.0) Requirement already satisfied: setuptools in ./venv-grpc/lib/python3.7/site-packages (from protobuf>=3.5.0.post1->grpcio-tools) (45.1.0) Requirement already satisfied: six>=1.9 in ./venv-grpc/lib/python3.7/site-packages (from protobuf>=3.5.0.post1->grpcio-tools) (1.14.0) Installing collected packages: protobuf, grpcio-tools Successfully installed grpcio-tools-1.26.0 protobuf-3.11.2
# note if I try to compile at first it will fail because the gnmi_ext file is in another folder and referenced by url.
# I just copied the gnmi_ext content to the gnmi folder and imported the filename only.
# clone gnmi proto repository
(venv-grpc)$ git clone https://github.com/openconfig/gnmi
Cloning into 'gnmi'...
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 989 (delta 4), reused 18 (delta 1), pack-reused 961
Receiving objects: 100% (989/989), 492.49 KiB | 1.44 MiB/s, done.
Resolving deltas: 100% (455/455), done.
# failed attempt with original proto
(venv-grpc)$ python -m grpc_tools.protoc -I . --python_out=. gnmi/gnmi.proto github.com/openconfig/gnmi/proto/gnmi_ext/gnmi_ext.proto: File not found. gnmi/gnmi.proto:20:1: Import "github.com/openconfig/gnmi/proto/gnmi_ext/gnmi_ext.proto" was not found or had errors. gnmi/gnmi.proto:214:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:241:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:344:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:363:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:411:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:423:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:432:12: "gnmi_ext.Extension" is not defined. gnmi/gnmi.proto:444:12: "gnmi_ext.Extension" is not defined.
# successful compilation after import edit
(venv-grpc)$ python -m grpc_tools.protoc -I . --python_out=. ./gnmi.proto (venv-grpc)$ ls -l total 464 -rw-r--r-- 1 brusilva staff 89684 Jan 19 13:07 gnmi.pb.go -rw-r--r-- 1 brusilva staff 21803 Jan 19 13:11 gnmi.proto -rw-r--r-- 1 brusilva staff 12708 Jan 19 13:12 gnmi_ext.pb.go -rw-r--r-- 1 brusilva staff 2624 Jan 19 13:12 gnmi_ext.proto -rw-r--r-- 1 brusilva staff 10172 Jan 19 13:12 gnmi_ext_pb2.py -rw-r--r-- 1 brusilva staff 83 Jan 19 13:12 gnmi_ext_pb2_grpc.py -rw-r--r-- 1 brusilva staff 76035 Jan 19 13:14 gnmi_pb2.py
-rw-r--r-- 1 brusilva staff 4864 Jan 19 13:07 gnmi_pb2_g
There you go! Now you see these python files created. They are ready to be used now.
Now all you need is to import them into your python code.
(venv-grpc)$ python Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from gnmi.gnmi_pb2_grpc import gNMIStub >>> from gnmi.gnmi_pb2 import GetRequest, GetResponse, Path, PathElem, CapabilityRequest, Encoding, SetRequest, Update, TypedValue
Router configuration
Following is the basic gRPC config needed:
grpc port 57400 address-family dual <<< to enable for ipv4 and ipv6 max-request-total 256 max-request-per-user 32
Coding
Certificate and credentials
As part of the authentication process, you will need the pem file (certificate) from the router(s) you want to communicate with. Download the following file from the router(s): /misc/config/grpc/ems.pem.
Variable-wise, we will use the following bare bones:
with open("ems.pem", "rb") as fp: <<< make sure to read the pem as bytes string
pem = fp.read()
host = "10.8.70.51"
port = "57400" <<< this is the port defined in the gRPC config on the router
metadata=[('username', "username"), ('password', "password")]
options=[('grpc.ssl_target_name_override', 'ems.cisco.com'), ('grpc.max_receive_message_length', 1000000000)]
The options variable defines what url to be used for the ssl verification, which for Cisco should be ems.cisco.com, and that is needed for any IOS-XR router.
The message_length we are changing from the 4MB default.
You can check all the options available in this url: https://grpc.github.io/grpc/core/group__grpc__arg__keys.html
The rest is self-explanatory.
Sending a GetRequest
Before we actually establish the connection, let's first read the proto file to understand what we need to do in case we want to send a GetRequest for example.
https://github.com/openconfig/gnmi/blob/master/proto/gnmi/gnmi.proto
// GetRequest is sent when a client initiates a Get RPC. It is used to specify // the set of data elements for which the target should return a snapshot of // data. The use_models field specifies the set of schema modules that are to // be used by the target - where use_models is not specified then the target // must use all schema models that it has. // Reference: gNMI Specification Section 3.3.1 message GetRequest { Path prefix = 1; // Prefix used for paths. repeated Path path = 2; // Paths requested by the client. // Type of elements within the data tree. enum DataType { ALL = 0; // All data elements. CONFIG = 1; // Config (rw) only elements. STATE = 2; // State (ro) only elements. // Data elements marked in the schema as operational. This refers to data // elements whose value relates to the state of processes or interactions // running on the device. OPERATIONAL = 3; } DataType type = 3; // The type of data being requested. Encoding encoding = 5; // Encoding to be used. repeated ModelData use_models = 6; // The schema models to be used. // Extension messages associated with the GetRequest. See the // gNMI extension specification for further definition. repeated gnmi_ext.Extension extension = 7; }
As you can see, in order to send a request, we basically need:
1. prefix of type Path - basically what we are polling.
2. a list of path items of type Path, which is what we are requesting.
3. type of type DataType. This is what type of data we want. We have 4 options basically.
4. encoding of type Encoding. This specifies how we will serialize the data over the gRPC connection. As of this article, we support JSON_IETF and ASCII format.
So in every GetRequest, I have to specify the above as a minimum.
So 1st step is to actually create a credential with the pem file:
credentials = grpc.ssl_channel_credentials(pem) # object type is grpc.ChannelCredentials
Now we are ready to establish the connection.
try:
channel = grpc.secure_channel(':'.join([host, port]), credentials, options) <<< create the secure channel
grpc.channel_ready_future(channel).result(timeout=10) <<< we specify the timeout as 10s and provide the channel
gnmi_stub = gNMIStub(channel) <<< creating the stub object and providing the channel
get_path = create_gnmi_path("Cisco-IOS-XR-ip-tcp-cfg:ip-tcp") <<< in this example we are polling the ip-tcp config
get_message = GetRequest(path=[get_path], type=1, encoding=4) <<< paths are passed as a List. type 1 = CONFIG, encoding 4 is JSON_IETF
print(gnmi_stub.Get(get_message, metadata=metadata)) <<< printing the raw response
except grpc.FutureTimeoutError as e: <<< always check for TimeoutError
print(e)
print("Failed to connect")
Note that this create_gnmi_path function we created to just convert the model specified in green into a request with the following format:
path {
elem {
name: "Cisco-IOS-XR-ip-tcp-cfg:ip-tcp"
}
}
encoding: JSON_IETF
Note that while this Path looks like json, it has to be formatted and created using the classes based on the proto file. Here's how a Path message looks like:
message Path {
// Elements of the path are no longer encoded as a string, but rather within
// the elem field as a PathElem message.
repeated string element = 1 [deprecated=true];
string origin = 2; // Label to disambiguate path.
repeated PathElem elem = 3; // Elements of the path.
string target = 4; // The name of the target
// (Sec. 2.2.2.1)
}
origin and target are not necessary in this case.
What we need is the list of PathElem objects elem. In our case we are only using one path, but know you can do multiple.
Each PathElem is defined as follows:
message PathElem {
string name = 1; // The name of the element in the path.
map<string, string> key = 2; // Map of key (attribute) name to value.
}
so you can create one PathElem like this for example, and then put that into a list, and lastly you can create the Path object:
# create elem
elem_name = 'Cisco-IOS-XR-ip-tcp-cfg:ip-tcp' path_elem = PathElem(name=elem_name, key={})
# append elem to a list as defined in proto file
list_of_path_elems = []
list_of_path_elems.append(path_elem)
# create Path object
p = Path(elem=list_of_path_elems)
So then when you send a Get, you will send "p" object.
This is how we get the response:
notification { <<< every response in gNMI is returned in a notification protocol buffer
timestamp: 1575572561137134733 <<< always timestamped. nanoseconds since epoch.
update { <<< update is a list of items
path {
elem {
name: "Cisco-IOS-XR-ip-tcp-cfg:ip-tcp" <<< name is the model we are polling.
}
}
val { <<< within val is our response
json_ietf_val: "{\"path-mtu-discovery\":10,\"syn-wait-time\":5}" <<< encoding type is json_ietf, so we get it in that format
}
}
}
error {
}
So you can compare and get used to the proto buff language definition syntax, here's how the notification message looks like from the gnmi proto file below.
Just like the response we get, we have timestamp and prefix (path). And the update is defined as a "repeated" (or "list" in python).
message Notification {
int64 timestamp = 1; // Timestamp in nanoseconds since Epoch.
Path prefix = 2; // Prefix used for paths in the message.
// An alias for the path specified in the prefix field.
// Reference: gNMI Specification Section 2.4.2
string alias = 3;
repeated Update update = 4; // Data elements that have changed values.
repeated Path delete = 5; // Data elements that have been deleted.
// This notification contains a set of paths that are always updated together
// referenced by a globally unique prefix.
bool atomic = 6;
}
Note we also have the response itself that comes as a val key. Look at the proto file for the update message type:
message Update {
Path path = 1; // The path (key) for the update.
Value value = 2 [deprecated=true]; // The value (value) for the update.
TypedValue val = 3; // The explicitly typed update value.
uint32 duplicates = 4; // Number of coalesced duplicates.
}
The encoding is of type json_ietf (we support only that and ascii), and the response comes as bytes string that you can convert to dictionary. See below how the proto file defines the TypedValue message type:
message TypedValue {
// One of the fields within the val oneof is populated with the value
// of the update. The type of the value being included in the Update
// determines which field should be populated. In the case that the
// encoding is a particular form of the base protobuf type, a specific
// field is used to store the value (e.g., json_val).
oneof value {
string string_val = 1; // String value.
int64 int_val = 2; // Integer value.
uint64 uint_val = 3; // Unsigned integer value.
bool bool_val = 4; // Bool value.
bytes bytes_val = 5; // Arbitrary byte sequence value.
float float_val = 6; // Floating point value.
Decimal64 decimal_val = 7; // Decimal64 encoded value.
ScalarArray leaflist_val = 8; // Mixed type scalar array value.
google.protobuf.Any any_val = 9; // protobuf.Any encoded bytes.
bytes json_val = 10; // JSON-encoded text.
bytes json_ietf_val = 11; // JSON-encoded text per RFC7951. <<< this is the type we are getting string ascii_val = 12; // Arbitrary ASCII text.
// Protobuf binary encoded bytes. The message type is not included.
// See the specification at
// github.com/openconfig/reference/blob/master/rpc/gnmi/protobuf-vals.md
// for a complete specification.
bytes proto_bytes = 13;
}
}
Polling Operational Data
When you want to use gnmi for get-oper, you need not only the yang model but also the top-level containers. If you run a GetCapability to the router, you only get the models but not the top-level containers.
For production scenarios, ideally you will have a list of models + paths you want to pull, and then just run a get for each.
You can also specify all models at once. In gRPC terms each model is a
You should then be able to loop through everything you defined if you wanted to get all oper models in a file for example.
for model in models: <<< models would be a list of paths
get_path = create_gnmi_path(model)
get_message = GetRequest(path=[get_path], type=3, encoding=4) <<< type 3 = OPERATIONAL
try:
out = gnmi_stub.Get(get_message, metadata=metadata)
except Exception as e:
print(e)
print(model)
This is just a simple example. Do it however you like
Polling Configuration Data
Partial Configuration
In order to pull config data via gnmi, you have the option to specify the model and top-level containers for the model in question - that's 1 way of doing it.
Here's an example, note this is just a snippet of the actual code:
models = ['Cisco-IOS-XR-ipv4-bgp-cfg:bgp'] <<< specify model:top-level-container in a list
for model in models:
get_path = create_gnmi_path(model)
get_message = GetRequest(path=[get_path],type=GetRequest.DataType.Value("CONFIG"), encoding=Encoding.Value("JSON_IETF"))
try:
out = gnmi_stub.Get(get_message, metadata=metadata)
print(f'Successfully got {model}\n\n{out}')
except Exception as e:
print(f'Error polling {model}. Exception:\n{e}')
And the response:
notification {
timestamp: 1575598106554876261
update {
path {
elem {
name: "Cisco-IOS-XR-ipv4-bgp-cfg:bgp"
}
}
val {
json_ietf_val: "{\"instance\":[{\"instance-name\":\"default\",\"instance-as\":[{\"as\":0,\"four-byte-as\":[{\"as\":100,\"bgp-running\":[null],\"default-vrf\":{\"global\":{\"router-id\":\"1.1.1.5\",\"graceful-restart-time\":120,\"graceful-restart-stalepath-time\":360,\"graceful-restart\":[null],\"neighbor-logging-detail\":[null],\"best-path-med-always\":[null],\"best-path-router-id\":[null],\"enforce-ibgp-out-policy\":[null],\"global-afs\":{\"global-af\":[{\"af-name\":\"ipv4-unicast\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\",\"attribute-download\":[null],\"ebgp\":{\"paths-value\":32,\"unequal-cost\":false,\"selective\":false,\"order-by-igp-metric\":false},\"ibgp\":{\"paths-value\":32,\"unequal-cost\":false,\"selective\":false,\"order-by-igp-metric\":false}},{\"af-name\":\"vpnv4-unicast\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\"},{\"af-name\":\"ipv6-unicast\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\",\"ebgp\":{\"paths-value\":32,\"unequal-cost\":false,\"selective\":false,\"order-by-igp-metric\":false},\"ibgp\":{\"paths-value\":32,\"unequal-cost\":false,\"selective\":false,\"order-by-igp-metric\":false}},{\"af-name\":\"vpnv6-unicast\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\"},{\"af-name\":\"l2vpn-vpls\",\"enable\":[null]},{\"af-name\":\"ipv4-mvpn\",\"enable\":[null],\"additional-paths-receive\":\"enable\",\"additional-paths-send\":\"enable\"}]}},\"bgp-entity\":{\"neighbor-groups\":{\"neighbor-group\":[{\"neighbor-group-name\":\"IBGP-RR\",\"create\":[null],\"remote-as\":{\"as-xx\":0,\"as-yy\":100},\"update-source-interface\":\"Loopback0\",\"neighbor-group-afs\":{\"neighbor-group-af\":[{\"af-name\":\"ipv4-unicast\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"vpnv4-unicast\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"ipv6-unicast\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"vpnv6-unicast\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"l2vpn-vpls\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}},{\"af-name\":\"ipv4-mvpn\",\"activate\":[null],\"maximum-prefixes\":{\"prefix-limit\":4294967295,\"warning-percentage\":75,\"warning-only\":false,\"restart-time\":0,\"discard-extra-paths\":false},\"soft-reconfiguration\":{\"inbound-soft\":true,\"soft-always\":true}}]}}]},\"neighbors\":{\"neighbor\":[{\"neighbor-address\":\"1.1.1.1\",\"neighbor-group-add-member\":\"IBGP-RR\",\"description\":\"RR-II11-5501-Bran\"},{\"neighbor-address\":\"1.1.1.6\",\"neighbor-group-add-member\":\"IBGP-RR\",\"description\":\"RR-II10-XRv9k-Varys\"}]}}},\"vrfs\":{\"vrf\":[{\"vrf-name\":\"spoke2\",\"vrf-global\":{\"exists\":[null],\"route-distinguisher\":{\"type\":\"as\",\"as-xx\":0,\"as\":200,\"as-index\":200},\"vrf-global-afs\":{\"vrf-global-af\":[{\"af-name\":\"ipv4-unicast\",\"enable\":[null],\"connected-routes\":{\"default-metric\":1}}]}},\"vrf-neighbors\":{\"vrf-neighbor\":[{\"neighbor-address\":\"5.20.0.2\",\"remote-as\":{\"as-xx\":0,\"as-yy\":65002},\"vrf-neighbor-afs\":{\"vrf-neighbor-af\":[{\"af-name\":\"ipv4-unicast\",\"activate\":[null],\"route-policy-in\":\"pass\",\"route-policy-out\":\"pass\"}]}}]}},{\"vrf-name\":\"mvpn_p10_vrf1\",\"vrf-global\":{\"exists\":[null],\"route-distinguisher\":{\"type\":\"as\",\"as-xx\":0,\"as\":10,\"as-index\":9},\"vrf-global-afs\":{\"vrf-global-af\":[{\"af-name\":\"ipv4-unicast\",\"enable\":[null],\"connected-routes\":{}},{\"af-name\":\"ipv4-mvpn\",\"enable\":[null]}]}}}]}}]}]}]}"
}
}
}
error {
}
Same way as polling a operational model, the response itself is inside the json_ietf_val key.
Full Configuration
What if you want to pull the full-config? Very easy! Just specify an empty Path object. Like this:
get_path = create_gnmi_path(model)
get_message = GetRequest(path=[Path()], type=GetRequest.DataType.Value("CONFIG"), encoding=Encoding.Value("JSON_IETF"))
try:
out = gnmi_stub.Get(get_message, metadata=metadata)
except Exception as e:
print(f'Error during get-config. Exception:\n{e}')
Then just use out.full_config to see the config. it is returned as a json/dict object already.
Changing Configuration
We use a SetRequest message type to change any configuration via gNMI.
// SetRequest is sent from a client to the target to update values in the data
// tree. Paths are either deleted by the client, or modified by means of being
// updated, or replaced. Where a replace is used, unspecified values are
// considered to be replaced, whereas when update is used the changes are
// considered to be incremental. The set of changes that are specified within
// a single SetRequest are considered to be a transaction.
// Reference: gNMI Specification Section 3.4.1
message SetRequest {
Path prefix = 1; // Prefix used for paths in the message.
repeated Path delete = 2; // Paths to be deleted from the data tree.
repeated Update replace = 3; // Updates specifying elements to be replaced.
repeated Update update = 4; // Updates specifying elements to updated.
// Extension messages associated with the SetRequest. See the
// gNMI extension specification for further definition.
repeated gnmi_ext.Extension extension = 5;
}
So we need the following information to change a config:
1. prefix = that is the path, or model we want to touch. It is a Path object.
2. update or replace = that is the list of updates or replacements we have - this is where your config will actually be stored. It is a list because you can change several different configs in a single transaction. It is a Update object.
In order to create an Update object, let's review the message format.
// Update is a re-usable message that is used to store a particular Path,
// Value pair.
// Reference: gNMI Specification Section 2.1
message Update {
Path path = 1; // The path (key) for the update.
Value value = 2 [deprecated=true]; // The value (value) for the update.
TypedValue val = 3; // The explicitly typed update value.
uint32 duplicates = 4; // Number of coalesced duplicates.
}
In here we need:
1. A Path object called path, which contains the model we want to change.
2. a TypedValue object val which has the config we want to replace or update.
Now that we know what we need, take a look at the config example below and sample code:
{
"interface-configuration": [{
"active": "act",
"interface-name": "Loopback10",
"interface-virtual": [
null
],
"Cisco-IOS-XR-ipv4-io-cfg:ipv4-network": {
"addresses": {
"primary": {
"address": "1.1.1.12",
"netmask": "255.255.255.255"
}
}
},
"Cisco-IOS-XR-ipv6-ma-cfg:ipv6-network": {
"addresses": {
"regular-addresses": {
"regular-address": [{
"address": "2001:1:1:1::12",
"prefix-length": 128,
"zone": "0"
}]
}
}
}
}]
}
The easiest way to get that config is to pull (get-config), make some modifications as you'd like, and push it (set). In this case, I will add a new loopback10 interface.
Note that the top-level model is not specified here. this will be in the Update object we create.
See the code below now:
# first we open the config file
config_json: dict = self.read_config(config_file) # define the path string, which is the model we are targeting
path: str = "Cisco-IOS-XR-ifmgr-cfg:interface-configurations" # then create Path object with path from filename
path_object: Path = self._create_gnmi_path(path) # The config in json is then used to create the TypedValue object in json_ietf_val format.
type_value: TypedValue = TypedValue(json_ietf_val=config_json) # Create Update message
update_object: Update = Update(path=path_object, val=type_value) # Here we do an update or replace based on the request.
if replace_or_update == "replace": set_request_object: SetRequest = SetRequest(replace=[update_object]) else: set_request_object: SetRequest = SetRequest(update=[update_object]) print(f'set_request_object: {set_request_object}') # Create the gnmi_stub to open the channel and send the Set. Store response in "response" which is type SetResponse self.gnmi_stub: gNMIStub = gNMIStub(self.channel) response: SetResponse = self.gnmi_stub.Set(set_request_object, metadata=self.metadata)
Subscription
Another feature of gRPC is the ability to subscribe to a certain Path and specify a desired interval. This is a great feature to allow the client to define what it wants from the router, without having to configure anything on it - all the information is in the application!
I am going to use the same bare bones script as above, just modifying to what we need - same variable names and all.
So while I show you how to code this, let's also learn the new message types we are going to handle.
I will handle this backwards, as we have multiple different message types and I think it's easier to go that way to understand first.
So the RPC itself is Subscribe, which gets a SubscribeRequest and returns a SubscribeResponse:
rpc Subscribe(stream SubscribeRequest) returns (stream SubscribeResponse);
The way I am processing the SubscribeResponse's is by creating a generator to walk over the "models" i am interested in. Like so:
def subscribe_to_path(request):
yield request
# stub is already created here
for response in gnmi_stub.Subscribe(subscribe_to_path(subscribe_request), metadata=gnmi_metadata):
print(response)
Let's look at the SubscribeRequest message format:
message SubscribeRequest {
oneof request {
SubscriptionList subscribe = 1; // Specify the paths within a subscription.
Poll poll = 3; // Trigger a polled update.
AliasList aliases = 4; // Aliases to be created.
}
// Extension messages associated with the SubscribeRequest. See the
// gNMI extension specification for further definition.
repeated gnmi_ext.Extension extension = 5;
}
So in our case, we will looking at a SubscriptionList. You can also specify a polled update.
A SubscribeRequest asks for a SubscriptionList. That means all models you specify in a SubscriptionList are considered to be a part of one single Subscription. Let's create then a SubscribeRequest:
# we provide the subscriptionlist to create a SubscribeRequest message
subscribe_request = SubscribeRequest(subscribe=subscriptionlist)
So now we need to create the SubscriptionList, which in my case is subscriptionlist. Below is the message format:
message SubscriptionList {
Path prefix = 1; // Prefix used for paths.
repeated Subscription subscription = 2; // Set of subscriptions to create.
// Whether target defined aliases are allowed within the subscription.
bool use_aliases = 3;
QOSMarking qos = 4; // DSCP marking to be used.
// Mode of the subscription.
enum Mode {
STREAM = 0; // Values streamed by the target (Sec. 3.5.1.5.2).
ONCE = 1; // Values sent once-off by the target (Sec. 3.5.1.5.1).
POLL = 2; // Values sent in response to a poll request (Sec. 3.5.1.5.3).
}
Mode mode = 5;
// Whether elements of the schema that are marked as eligible for aggregation
// should be aggregated or not.
bool allow_aggregation = 6;
// The set of schemas that define the elements of the data tree that should
// be sent by the target.
repeated ModelData use_models = 7;
// The encoding that the target should use within the Notifications generated
// corresponding to the SubscriptionList.
Encoding encoding = 8;
// An optional field to specify that only updates to current state should be
// sent to a client. If set, the initial state is not sent to the client but
// rather only the sync message followed by any subsequent updates to the
// current state. For ONCE and POLL modes, this causes the server to send only
// the sync message (Sec. 3.5.2.3).
bool updates_only = 9;
}
The path will be part of the Subscription - we will look at that next.
The Subscription messages are defined in a list, in order to allow us to specify multiple models in the same Subscription.
subscriptionlist_mode = SubscriptionList.Mode.Value("STREAM")
subscriptionlist_encoding = Encoding.Value("PROTO")
subscriptionlist = SubscriptionList(subscription=subscriptions,
mode=subscriptionlist_mode,
encoding=subscriptionlist_encoding)
Note the mode and encoding. We are using STREAM mode, with PROTO encoding. You are welcome to use JSON as well, but PROTO is leaner especially with high amounts of data.
You can use the following modes:
enum Mode {
STREAM = 0; // Values streamed by the target (Sec. 3.5.1.5.2).
ONCE = 1; // Values sent once-off by the target (Sec. 3.5.1.5.1).
POLL = 2; // Values sent in response to a poll request (Sec. 3.5.1.5.3).
}
Now onto the last message type we need to know - Subscription:
message Subscription {
Path path = 1; // The data tree path.
SubscriptionMode mode = 2; // Subscription mode to be used.
uint64 sample_interval = 3; // ns between samples in SAMPLE mode.
// Indicates whether values that have not changed should be sent in a SAMPLE
// subscription.
bool suppress_redundant = 4;
// Specifies the maximum allowable silent period in nanoseconds when
// suppress_redundant is in use. The target should send a value at least once
// in the period specified.
uint64 heartbeat_interval = 5;
}
Path we already know how to build, and mode can be one of the following:
enum SubscriptionMode {
TARGET_DEFINED = 0; // The target selects the relevant mode for each element.
ON_CHANGE = 1; // The target sends an update on element value change.
SAMPLE = 2; // The target samples values according to the interval.
}
You can leave it up to the server to define the type of subscription (TARGED_DEFINED), use ON_CHANGE (i.e. for administrative status of an interface), or SAMPLE, which is always going to dump the values in an interval specified.
Also note here that the sample_interval is in nanoseconds! Make sure to not try to subscribe to a huge model every 10ns
Now onto how we actually instantiate this message. In my case I am using two Subscription messages. That means 1 SubscribeRequest will ask for data about two paths specified in "models".
# Note we are doing what's called leaf-level streaming. This is supported on 721 onwards
models = ["openconfig-interfaces:interfaces/interface[name=TenGigE0/0/0/18/0]/state/counters/out-octets", "openconfig-interfaces:interfaces/interface[name=TenGigE0/0/0/18/1]/state/counters/out-octets"] # I personally like to specify the interval always in seconds and multiply after
sample_interval = 30 sample_rate = sample_interval*1000000000 # subscribe mode is just a string
subscribe_mode = "SAMPLE" subscriptions = [] for model in models:
# create the Path object sub_path = create_gnmi_path(model)
# create the SubscriptionMode object sub_mode = SubscriptionMode.Value(subscribe_mode)
# create the Subscription sub_subscription = Subscription(path=sub_path, mode=sub_mode, sample_interval=sample_rate) subscriptions.append(sub_subscription)
Now we got all the pieces! Remember the chain looks like this:
Subscription --> SubscriptionList --> SubscribeRequest --> Subscribe
SAMPLE
Let's actually run it on a router in SAMPLE mode and see what we get. I put the time below so you can see what happens.
-- 20:06:53 --
# This is the first response from the router - the 1st model
update {
timestamp: 1588464413328000000
prefix {
origin: "openconfig-interfaces"
elem {
name: "interfaces"
}
elem {
name: "interface"
key {
key: "name"
value: "TenGigE0/0/0/18/0"
}
}
elem {
name: "state"
}
elem {
name: "counters"
}
}
update {
path {
elem {
name: "out-octets"
}
}
val {
uint_val: 226343920640
}
}
}
-- 20:06:53 --
# This is the 2nd response - to the 2nd model
update {
timestamp: 1588464413708000000
prefix {
origin: "openconfig-interfaces"
elem {
name: "interfaces"
}
elem {
name: "interface"
key {
key: "name"
value: "TenGigE0/0/0/18/1"
}
}
elem {
name: "state"
}
elem {
name: "counters"
}
}
update {
path {
elem {
name: "out-octets"
}
}
val {
uint_val: 243752831062
}
}
}
-- 20:06:55 --
# This is a sync message after the subscriptions were all received
sync_response: true
What you notice here is. The update dictionary contains an update dictionary with the actual response of the out-octets from the router. It contains the path element name, as well as the value in unsigned integer as expected.
You will see now that the session is always going to be up, and every <interval>, you will get a new "SAMPLE" of the models you requested. Pretty cool, right?!
All you need to do after this is format the data as you wish for the database of your preference and start graphing it out!
ON_CHANGE
Now say you want to just get updates for a leaf or container when there are changes. You don't really need to poll at a certain interval - just keep the subscription established and when you subscribe, use the ON_CHANGE mode instead of SAMPLE.
Your subscription message would look like this for the following model:
# openconfig-interfaces:interfaces/interface[name=TenGigE0/0/0/18/0]/state/admin-status
subscription { path { elem { name: "openconfig-interfaces:interfaces" } elem { name: "interface" key { key: "name" value: "TenGigE0/0/0/18/0" } } elem { name: "state" } elem { name: "admin-status" } } mode: ON_CHANGE sample_interval: 10000000000 } encoding: PROTO
Immediately when the admin-status of this interface changes to any of the states defined in the model, you will receive a new update! That's the best way to deal with somewhat static data!
One important note - you need to use the following cli if you wish to receive update only about the leaf you are subscribing to, vs the whole bag or gathering point for the model.
telemetry model-driven include select-leaves-on-events
This is by default how TARGET_DEFINED works, which is described below.
TARGET_DEFINED
Let's say you don't know the best option, whether ON_CHANGE or SAMPLE for a particular leaf. You can use TARGET_DEFINED to allow the router to decide how to send the data, whether at SAMPLE mode or ON_CHANGE mode. Your Subscription would look like below - note that the mode is unspecified.
subscription {
path {
elem {
name: "openconfig-interfaces:interfaces"
}
elem {
name: "interface"
key {
key: "name"
value: "TenGigE0/0/0/18/0"
}
}
elem {
name: "state"
}
elem {
name: "admin-status"
}
}
sample_interval: 10000000000
}
API
In order to simplify the usage of GNMI using python scripts, I've created a simple library for GNMI Subscriptions:
