In this third part of the story porting an existing service to use a NETCONF NED, we'll encounter some real life hurdles. The NETCONF interface on the IOS-XE device I'm using is one of the best on the market, still there are a few caveats that I'll show how to work around. A couple of when statements also need to be added because the YANG model published by the device is different than the CLI YANG model.
In previous posts, we prepared the environment and made sure we have both a CLI NED and a NETCONF NED ready to go for our IOS-XE device, and then we sent a CLI configuration to the device, and retrieved the config diff over NETCONF/XML, thereby getting it translated to the YANG model used by the NETCONF interface. The XML diff was used as the basis for an updated version of the device template. You don't have to go back to read the previous post if you don't want to, but if you do, it's here Porting an existing service to use a NETCONF NED, part 2
Call me a liar, but I wanted to have a nice, clean end of the previous post. That post demonstrates in principle how you translate a CLI NED template to a NETCONF NED template. Except that the template I arrived at there doesn't work. Or, more specifically, what I showed to be working does work, I just didn't show what doesn't, and I pretended everything was fine.
Here's what happens. When I commit the service, it commits fine. Actually, NSO thinks everything *is* fine, so nothing is reported back to the operator. But when we run a compare-config, we can see that the configuration on the device doesn't actually match what NSO ordered it to be. Someone/something has changed it:
JLINDBLA-M-W0J2(config)# commit
Commit complete.
JLINDBLA-M-W0J2(config)# devices device ce0 compare-config
diff
devices {
device ce0 {
config {
ios:router {
bgp 61000 {
bgp {
+ log-neighbor-changes true;
}
- network 10.0.0.0 {
- }
neighbor 192.168.1.2 {
- activate;
}
}
}
}
}
}
There is an object log-neighbor-changes that has appeared spontaneously. After speaking to the XE team, I learned this is a default config that appears automatically. That would be all fine if it was modeled as such in the YANG, but it isn't. So from NSO's perspective, this is some additional config that someone else (well, the device in this case) has created out-of band (OOB) after NSO created the bgp 61000 instance.
Furthermore, the network 10.0.0.0 is nowhere to be seen. What happened there? Again, the XE team provides the explanation. The XE CLI allows network and neighbor to be given directly under bgp 61000 if they pertain to address-family ipv4 unicast. Legacy reasons. For any other address family, you'd have to go to the respective address-family submode and set them there.
Finally, the leaf activate under neighbor 192.168.1.2 is missing. This is a plain NETCONF bug, explains the XE team. It's not being reported in NETCONF <get-config> as it should.
So what can be done about this? Actually, turns out (I've already seen the end of the story 
) all these issues can be worked around so that the service works perfectly. First thing to do is to make sure the service creates that log-neighbor-changes element. If we ask for it to be created, that will hide the fact that the YANG model is missing a default. The next work around is to switch to using neighbor and network under the proper address family (ipv4 unicast). Look at how the neighbor and network are now under address-family/no-vrf/ipv4:
<router>
<bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-bgp">
<id>{/as-number}</id>
<bgp>
<log-neighbor-changes/>
</bgp>
<address-family>
<no-vrf>
<ipv4>
<af-name>unicast</af-name>
<neighbor>
<id>{$LINK_PE_ADR}</id>
<remote-as>100</remote-as>
<activate/>
</neighbor>
<network>
<number>{$LOCAL_CE_NET}</number>
</network>
</ipv4>
</no-vrf>
</address-family>
</bgp>
</router>
With the updated template, reload packages, redeploying the service works fine, but compare-config reveals that our work arounds still aren't sufficient.
JLINDBLA-M-W0J2(config)# devices device ce0 compare-config
diff
devices {
device ce0 {
config {
ios-native:native {
router {
bgp 61000 {
+ # first
+ neighbor 192.168.1.2 {
+ remote-as 100;
+ }
address-family {
no-vrf {
ipv4 unicast {
neighbor 192.168.1.2 {
- remote-as 100;
}
}
}
}
}
}
}
}
}
}
First of all, moving our neighbor into the address family seems to have worked, but now it shows up *twice*. It's visible once (unexpectedly) outside the address family, and once (as expected) inside. Furthermore, the remote-as setting is visible in the data outside address-family, but not in the one inside (bug!). What to do? Well, if we make our template match the device behavior, pretending that this is what we want, the issue will disappear.
So here I'll just copy the neighbor data, so that we ask for it twice, both inside and outside the address-family.
<router>
<bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-bgp">
<id>{/as-number}</id>
<bgp>
<log-neighbor-changes/>
</bgp>
<neighbor>
<id>{$LINK_PE_ADR}</id>
<remote-as>100</remote-as>
<activate/>
</neighbor>
<address-family>
<no-vrf>
<ipv4>
<af-name>unicast</af-name>
<neighbor>
<id>{$LINK_PE_ADR}</id>
<remote-as>100</remote-as>
<activate/>
</neighbor>
<network>
<number>{$LOCAL_CE_NET}</number>
</network>
</ipv4>
</no-vrf>
</address-family>
</bgp>
</router>
Try again, reload packages, re-deploy service, compare config. Seems we're getting closer, but still there's a diff:
JLINDBLA-M-W0J2(config)# devices device ce0 compare-config
diff
devices {
device ce0 {
config {
ios-native:native {
router {
bgp 61000 {
neighbor 192.168.1.2 {
- activate;
}
address-family {
no-vrf {
ipv4 unicast {
neighbor 192.168.1.2 {
- remote-as 100;
}
}
}
}
}
}
}
}
}
}
Ok, so the leaf activate seems not to work properly in the neighbor outside the address-family, but remote-as works there (not included in the diff). Inside the address-family it's the other way around. The leaf activate works there (not included in the diff), but remote-as is not working. Ok, I feel a bit ridiculous, but this is the template I have to use here:
<router>
<bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-bgp">
<id>{/as-number}</id>
<bgp>
<log-neighbor-changes/>
</bgp>
<neighbor>
<id>{$LINK_PE_ADR}</id>
<remote-as>100</remote-as>
</neighbor>
<address-family>
<no-vrf>
<ipv4>
<af-name>unicast</af-name>
<neighbor>
<id>{$LINK_PE_ADR}</id>
<activate/>
</neighbor>
<network>
<number>{$LOCAL_CE_NET}</number>
</network>
</ipv4>
</no-vrf>
</address-family>
</bgp>
</router>
Now when I reload and try again, it works. Zero diff!
JLINDBLA-M-W0J2(config)# comm
Commit complete.
JLINDBLA-M-W0J2(config)# devices device ce0 compare-config
JLINDBLA-M-W0J2(config)#
Ok, now that we have a basic working service over NETCONF, we have to extend it so that it covers all the features in all the templates used by the service. This is "more of the same" kind of work described previously, so I'll keep it short. The service instance I have been playing with is a very basic one, so it's covered by the single template we have already ported. But the service can do more. In particular, if I set a QoS parameter on the L3VPN service instance, the service will require the additional templates. Looking at the L3VPN service template directory, I find the following that relates to XE devices:
$ grep -rils urn:ios packages/l3vpn/templates/
packages/l3vpn/templates//l3vpn-acl.xml
packages/l3vpn/templates//l3vpn-ce.xml
packages/l3vpn/templates//l3vpn-pe.xml
packages/l3vpn/templates//l3vpn-qos-class.xml
packages/l3vpn/templates//l3vpn-qos-prio.xml
packages/l3vpn/templates//l3vpn-qos.xml
The l3vpn-ce.xml is what we already ported. The l3vpn-pe.xml is not relevant for the CE role. So four rather small templates remain to port. So let's repeat the work from the previous post. First, I will switch the device ce0 back to be a CLI device.
JLINDBLA-M-W0J2# con
Entering configuration mode terminal
JLINDBLA-M-W0J2(config)# devices device ce0 device-type cli ned-id cisco-ios
JLINDBLA-M-W0J2(config-device-ce0)# no port
JLINDBLA-M-W0J2(config-device-ce0)# comm
Commit complete.
JLINDBLA-M-W0J2(config-device-ce0)# top
JLINDBLA-M-W0J2(config)# devices sync-f
sync-result {
device ce0
result true
}
sync-result {
device ce0-nc
result true
}
sync-result {
device ce1
result true
Note that I synced ce0 (over CLI) as well as ce0-nc (over NETCONF) now, so we have a fresh baseline.
The service config I have running looks like this:
JLINDBLA-M-W0J2(config)# show full-configuration vpn
vpn l3vpn skoda
as-number 61000
endpoint newyork-office
ce-device ce3
ce-interface GigabitEthernet0/2
ip-network 10.1.0.0/24
bandwidth 2000000
!
endpoint riodejaneiro-office
ce-device ce6
ce-interface GigabitEthernet0/2
ip-network 10.0.0.0/24
bandwidth 10000000
!
endpoint tokyo-office
ce-device ce0
ce-interface GigabitEthernet2
ip-network 10.0.0.0/24
bandwidth 10000000
!
!
I now need to extend this to also use some QoS service parameters.
JLINDBLA-M-W0J2(config)# vpn l3vpn skoda qos qos-policy GOLD
JLINDBLA-M-W0J2(config-l3vpn-skoda)# commit dry-run
cli {
local-node {
data devices {
device ce0 {
config {
ios:ip {
access-list {
extended {
+ ext-named-acl GLOBAL-call-signaling {
+ ext-access-list-rule "permit tcp any any range 5060 5061";
+ }
+ ext-named-acl GLOBAL-ssh {
+ ext-access-list-rule "permit tcp any any range 22 22";
+ }
+ ext-named-acl GLOBAL-voice {
+ ext-access-list-rule "permit udp any any range 16348 32767";
+ }
}
}
}
+ ios:class-map BUSINESS-CRITICAL {
+ match-any;
+ match {
+ access-group {
+ name GLOBAL-ssh;
+ }
+ }
+ }
+ ios:class-map MISSION-CRITICAL {
+ match-any;
+ match {
+ access-group {
+ name GLOBAL-call-signaling;
+ }
+ }
+ }
...
JLINDBLA-M-W0J2(config-l3vpn-skoda)# commit
Commit complete.
JLINDBLA-M-W0J2(config-l3vpn-skoda)# top
The trick with showing the XML diff on the ce0-nc shadow device should be familiar by now.
JLINDBLA-M-W0J2(config)# devices device ce0-nc compare-config outformat xml
diff
<devices xmlns="http://tail-f.com/ns/ncs">
<device>
<name>ce0-nc</name>
<config>
<native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native">
<ip>
<access-list>
<extended xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-acl">
<name>GLOBAL-call-signaling</name>
<access-list-seq-rule>
<sequence>10</sequence>
<ace-rule>
<action>permit</action>
<protocol>tcp</protocol>
<any/>
<dst-any/>
<dst-range1>5060</dst-range1>
<dst-range2>5061</dst-range2>
</ace-rule>
</access-list-seq-rule>
</extended>
<extended xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-acl">
<name>GLOBAL-ssh</name>
<access-list-seq-rule>
<sequence>10</sequence>
<ace-rule>
<action>permit</action>
<protocol>tcp</protocol>
<any/>
<dst-any/>
<dst-eq>22</dst-eq>
</ace-rule>
</access-list-seq-rule>
</extended>
<extended xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-acl">
<name>GLOBAL-voice</name>
<access-list-seq-rule>
<sequence>10</sequence>
<ace-rule>
<action>permit</action>
<protocol>udp</protocol>
<any/>
<dst-any/>
<dst-range1>16348</dst-range1>
<dst-range2>32767</dst-range2>
</ace-rule>
</access-list-seq-rule>
</extended>
</access-list>
</ip>
<policy>
<class-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">
<name>BUSINESS-CRITICAL</name>
<prematch>match-any</prematch>
<match>
<access-group>
<name>GLOBAL-ssh</name>
</access-group>
</match>
</class-map>
<class-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">
<name>MISSION-CRITICAL</name>
<prematch>match-any</prematch>
<match>
<access-group>
<name>GLOBAL-call-signaling</name>
</access-group>
</match>
</class-map>
<class-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">
<name>REALTIME</name>
<prematch>match-any</prematch>
<match>
<access-group>
<name>GLOBAL-voice</name>
</access-group>
</match>
</class-map>
<policy-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">
<name>GOLD</name>
<class>
<name>BUSINESS-CRITICAL</name>
<action-list>
<action-type>bandwidth</action-type>
<bandwidth>
<percent>20</percent>
</bandwidth>
</action-list>
<action-list>
<action-type>set</action-type>
<set>
<ip>
<dscp>
<dscp-val>af21</dscp-val>
</dscp>
</ip>
</set>
</action-list>
</class>
<class>
<name>MISSION-CRITICAL</name>
<action-list>
<action-type>bandwidth</action-type>
<bandwidth>
<percent>25</percent>
</bandwidth>
</action-list>
<action-list>
<action-type>set</action-type>
<set>
<ip>
<dscp>
<dscp-val>af31</dscp-val>
</dscp>
</ip>
</set>
</action-list>
</class>
<class>
<name>REALTIME</name>
<action-list>
<action-type>priority</action-type>
<priority>
<percent>20</percent>
</priority>
</action-list>
<action-list>
<action-type>set</action-type>
<set>
<ip>
<dscp>
<dscp-val>ef</dscp-val>
</dscp>
</ip>
</set>
</action-list>
</class>
</policy-map>
<policy-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">
<name>skoda</name>
<class>
<name>class-default</name>
<action-list>
<action-type>service-policy</action-type>
<service-policy>GOLD</service-policy>
</action-list>
</class>
</policy-map>
</policy>
<router>
<bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-bgp">
<id>61000</id>
<neighbor>
<id>192.168.1.2</id>
<remote-as>100</remote-as>
</neighbor>
<address-family>
<no-vrf>
<ipv4>
<af-name>unicast</af-name>
<neighbor>
<id>192.168.1.2</id>
<activate/>
</neighbor>
</ipv4>
</no-vrf>
</address-family>
</bgp>
</router>
</native>
</config>
</device>
</devices>
Now, matching this towards the templates in the service package, like we did previously, I end up with new templates as follows:
l3vpn-qos.xml:
<native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" tags="merge">
<policy>
<policy-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">
<name>{/name}</name>
<class>
<name>class-default</name>
<action-list>
<action-type>service-policy</action-type>
<service-policy>{$POLICY_NAME}</service-policy>
</action-list>
</class>
</policy-map>
</policy>
</native>
l3vpn-qos-prio.xml:
<native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" tags="merge">
<policy>
<policy-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">
<name>{$POLICY_NAME}</name>
<class>
<name>{$CLASS_NAME}</name>
<action-list>
<action-type>priority</action-type>
<priority>
<percent>{$CLASS_BW}</percent>
</priority>
</action-list>
<action-list>
<action-type>set</action-type>
<set>
<ip>
<dscp>
<dscp-val>{$CLASS_DSCP}</dscp-val>
</dscp>
</ip>
</set>
</action-list>
</class>
</policy-map>
</policy>
</native>
l3vpn-qos-class.xml:
<native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" tags="merge">
<policy>
<class-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">
<name>{$CLASS_NAME}</name>
<prematch>match-any</prematch>
<match>
<access-group>
<name>{$MATCH_ENTRY}</name>
</access-group>
</match>
</class-map>
</policy>
</native>
l3vpn-acl.xml:
<native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" tags="merge">
<ip>
<access-list>
<extended xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-acl">
<name>{$ACL_NAME}</name>
<access-list-seq-rule>
<sequence>10</sequence>
<ace-rule>
<action>permit</action>
<protocol>{$PROTOCOL}</protocol>
<any/><!-- FIXME: {$SOURCE_IP_ADR} {$SOURCE_WMASK} -->
<dst-any/><!-- FIXME: {$DEST_IP_ADR} {$DEST_WMASK} -->
<dst-range1 when="{boolean($PORT_START) and boolean($PORT_END) and ($PORT_START!=$PORT_END)}">{$PORT_START}}</dst-range1>
<dst-range2 when="{boolean($PORT_START) and boolean($PORT_END) and ($PORT_START!=$PORT_END)}">{$PORT_END}</dst-range2>
<dst-eq when="{boolean($PORT_START) and (not($PORT_END) or ($PORT_END=$PORT_START))}">{$PORT_START}</dst-eq>
</ace-rule>
</access-list-seq-rule>
</extended>
</access-list>
</ip>
</native>
Here, in the ACL template, a few quirks are worth discussing. In the old CLI-based template, the ACL template was about constructing a string something like this: "premit tcp any any range 5060 5061". In the YANG model used over the NETCONF interface, this line has been split up into individual leafs (great!). When it comes to the port/port-range, I have to use two different constructs. dst-range1 .. dst-range2 if there is an actual range, and dst-eq if it's a single port. The application code could be updated to apply different templates for each case, but I'm good at XPATH, so I can construct some when expressions to ensure the right alternative is used depending on the service instance input data. This way I don't have to change any of the service code.
Also, the CLI-based template was made to inject the source and destination IP addresses in there. I didn't figure out what I needed to do in the service input data to make them actually be used, so I just commented out those XPATH variables with a FIXME. If anyone knows when they would be used, I could add that later.
So with these new templates, let's see what happens:
JLINDBLA-M-W0J2# con
Entering configuration mode terminal
JLINDBLA-M-W0J2(config)# vpn l3vpn skoda qos qos-policy GOLD
JLINDBLA-M-W0J2(config-l3vpn-skoda)# commit dry-run
cli {
local-node {
data devices {
device ce0 {
config {
ios-native:native {
ip {
access-list {
+ extended GLOBAL-call-signaling {
+ access-list-seq-rule 10 {
+ ace-rule {
+ action permit;
+ protocol tcp;
+ any;
+ dst-any;
+ dst-range1 5060;
+ dst-range2 5061;
+ }
+ }
+ }
+ extended GLOBAL-ssh {
+ access-list-seq-rule 10 {
+ ace-rule {
+ action permit;
+ protocol tcp;
+ any;
+ dst-any;
+ dst-eq 22;
+ }
+ }
+ }
+ extended GLOBAL-voice {
+ access-list-seq-rule 10 {
+ ace-rule {
+ action permit;
+ protocol udp;
+ any;
+ dst-any;
+ dst-range1 16348;
+ dst-range2 32767;
+ }
+ }
+ }
}
}
policy {
+ class-map BUSINESS-CRITICAL {
+ prematch match-any;
+ match {
+ access-group {
+ name [ GLOBAL-ssh ];
+ }
+ }
+ }
+ class-map MISSION-CRITICAL {
+ prematch match-any;
+ match {
+ access-group {
+ name [ GLOBAL-call-signaling ];
+ }
+ }
+ }
+ class-map REALTIME {
+ prematch match-any;
+ match {
+ access-group {
+ name [ GLOBAL-voice ];
+ }
+ }
+ }
+ policy-map GOLD {
+ class REALTIME {
+ action-list priority {
+ priority {
+ percent 20;
+ }
+ }
+ action-list set {
+ set {
+ ip {
+ dscp {
+ dscp-val ef;
+ }
+ }
+ }
+ }
+ }
+ }
policy-map skoda {
class class-default {
+ # after action-list shape
+ action-list service-policy {
+ service-policy GOLD;
+ }
}
}
}
}
}
}
device ce3 {
config {
ios:ip {
...
Looks good. Let's try the real test then:
JLINDBLA-M-W0J2(config-l3vpn-skoda)# comm
Commit complete.
JLINDBLA-M-W0J2(config-l3vpn-skoda)# top
JLINDBLA-M-W0J2(config)# devices device ce0 compare-config
JLINDBLA-M-W0J2(config)# vpn l3vpn skoda un-deploy
JLINDBLA-M-W0J2(config)# devices device ce0 compare-config
JLINDBLA-M-W0J2(config)# vpn l3vpn skoda re-deploy
JLINDBLA-M-W0J2(config)# devices device ce0 compare-config
JLINDBLA-M-W0J2(config)#
Yay! We're able to both apply and remove the service instance, and after each change there is zero diff. NSO and the device keeps in sync (with some workarounds in the service, sure).