cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements

Community Helping Community

Porting an existing service to use a NETCONF NED, part 3

586
Views
2
Helpful
0
Comments
Cisco Employee

In this third part of the story porting an existing service to use a NETCONF NED, we'll encounter some real life hurdles. The NETCONF interface on the IOS-XE device I'm using is one of the best on the market, still there are a few caveats that I'll show how to work around. A couple of when statements also need to be added because the YANG model published by the device is different than the CLI YANG model.

 

In previous posts, we prepared the environment and made sure we have both a CLI NED and a NETCONF NED ready to go for our IOS-XE device, and then we sent a CLI configuration to the device, and retrieved the config diff over NETCONF/XML, thereby getting it translated to the YANG model used by the NETCONF interface. The XML diff was used as the basis for an updated version of the device template. You don't have to go back to read the previous post if you don't want to, but if you do, it's here Porting an existing service to use a NETCONF NED, part 2

 

Call me a liar, but I wanted to have a nice, clean end of the previous post. That post demonstrates in principle how you translate a CLI NED template to a NETCONF NED template. Except that the template I arrived at there doesn't work. Or, more specifically, what I showed to be working does work, I just didn't show what doesn't, and I pretended everything was fine.

 

Here's what happens. When I commit the service, it commits fine. Actually, NSO thinks everything *is* fine, so nothing is reported back to the operator. But when we run a compare-config, we can see that the configuration on the device doesn't actually match what NSO ordered it to be. Someone/something has changed it:

 

JLINDBLA-M-W0J2(config)# commit       

Commit complete.

JLINDBLA-M-W0J2(config)# devices device ce0 compare-config

diff

devices {

     device ce0 {

         config {

             ios:router {

                 bgp 61000 {

                     bgp {

+                        log-neighbor-changes true;

                     }

-                    network 10.0.0.0 {

-                    }

                     neighbor 192.168.1.2 {

-                        activate;

                     }

                 }

             }

         }

     }

}

 

There is an object log-neighbor-changes that has appeared spontaneously. After speaking to the XE team, I learned this is a default config that appears automatically. That would be all fine if it was modeled as such in the YANG, but it isn't. So from NSO's perspective, this is some additional config that someone else (well, the device in this case) has created out-of band (OOB) after NSO created the bgp 61000 instance.

 

Furthermore, the network 10.0.0.0 is nowhere to be seen. What happened there? Again, the XE team provides the explanation. The XE CLI allows network and neighbor to be given directly under bgp 61000 if they pertain to address-family ipv4 unicast. Legacy reasons. For any other address family, you'd have to go to the respective address-family submode and set them there.

 

Finally, the leaf activate under neighbor 192.168.1.2 is missing. This is a plain NETCONF bug, explains the XE team. It's not being reported in NETCONF <get-config> as it should.

 

So what can be done about this? Actually, turns out (I've already seen the end of the story ) all these issues can be worked around so that the service works perfectly. First thing to do is to make sure the service creates that log-neighbor-changes element. If we ask for it to be created, that will hide the fact that the YANG model is missing a default. The next work around is to switch to using neighbor and network under the proper address family (ipv4 unicast). Look at how the neighbor and network are now under address-family/no-vrf/ipv4:

 

          <router>

            <bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-bgp">

              <id>{/as-number}</id>

              <bgp>

                <log-neighbor-changes/>

              </bgp>

              <address-family>

                <no-vrf>

                  <ipv4>

                    <af-name>unicast</af-name>

                    <neighbor>

                      <id>{$LINK_PE_ADR}</id>

                      <remote-as>100</remote-as>

                      <activate/>

                    </neighbor>

                    <network>

                      <number>{$LOCAL_CE_NET}</number>

                    </network>

                  </ipv4>

                </no-vrf>

              </address-family>

            </bgp>

          </router>

 

With the updated template, reload packages, redeploying the service works fine, but compare-config reveals that our work arounds still aren't sufficient.

 

JLINDBLA-M-W0J2(config)# devices device ce0 compare-config

diff

devices {

     device ce0 {

         config {

             ios-native:native {

                 router {

                     bgp 61000 {

+                        # first

+                        neighbor 192.168.1.2 {

+                            remote-as 100;

+                        }

                         address-family {

                             no-vrf {

                                 ipv4 unicast {

                                     neighbor 192.168.1.2 {

-                                        remote-as 100;

                                     }

                                 }

                             }

                         }

                     }

                 }

             }

         }

     }

}

 

First of all, moving our neighbor into the address family seems to have worked, but now it shows up *twice*. It's visible once (unexpectedly) outside the address family, and once (as expected) inside. Furthermore, the remote-as setting is visible in the data outside address-family, but not in the one inside (bug!). What to do? Well, if we make our template match the device behavior, pretending that this is what we want, the issue will disappear.

 

So here I'll just copy the neighbor data, so that we ask for it twice, both inside and outside the address-family.

 

          <router>

            <bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-bgp">

              <id>{/as-number}</id>

              <bgp>

                <log-neighbor-changes/>

              </bgp>

              <neighbor>

                <id>{$LINK_PE_ADR}</id>

                <remote-as>100</remote-as>

                <activate/>

              </neighbor>

              <address-family>

                <no-vrf>

                  <ipv4>

                    <af-name>unicast</af-name>

                    <neighbor>

                      <id>{$LINK_PE_ADR}</id>

                      <remote-as>100</remote-as>

                      <activate/>

                    </neighbor>

                    <network>

                      <number>{$LOCAL_CE_NET}</number>

                    </network>

                  </ipv4>

                </no-vrf>

              </address-family>

            </bgp>

          </router>

 

Try again, reload packages, re-deploy service, compare config. Seems we're getting closer, but still there's a diff:

 

JLINDBLA-M-W0J2(config)# devices device ce0 compare-config

diff

devices {

     device ce0 {

         config {

             ios-native:native {

                 router {

                     bgp 61000 {

                         neighbor 192.168.1.2 {

-                            activate;

                         }

                         address-family {

                             no-vrf {

                                 ipv4 unicast {

                                     neighbor 192.168.1.2 {

-                                        remote-as 100;

                                     }

                                 }

                             }

                         }

                     }

                 }

             }

         }

     }

}

 

Ok, so the leaf activate seems not to work properly in the neighbor outside the address-family, but remote-as works there (not included in the diff). Inside the address-family it's the other way around. The leaf activate works there (not included in the diff), but remote-as is not working. Ok, I feel a bit ridiculous, but this is the template I have to use here:

 

          <router>

            <bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-bgp">

              <id>{/as-number}</id>

              <bgp>

                <log-neighbor-changes/>

              </bgp>

              <neighbor>

                <id>{$LINK_PE_ADR}</id>

                <remote-as>100</remote-as>

              </neighbor>

              <address-family>

                <no-vrf>

                  <ipv4>

                    <af-name>unicast</af-name>

                    <neighbor>

                      <id>{$LINK_PE_ADR}</id>

                      <activate/>

                    </neighbor>

                    <network>

                      <number>{$LOCAL_CE_NET}</number>

                    </network>

                  </ipv4>

                </no-vrf>

              </address-family>

            </bgp>

          </router>

 

Now when I reload and try again, it works. Zero diff!

 

JLINDBLA-M-W0J2(config)# comm

Commit complete.

JLINDBLA-M-W0J2(config)# devices device ce0 compare-config             

JLINDBLA-M-W0J2(config)#

 

Ok, now that we have a basic working service over NETCONF, we have to extend it so that it covers all the features in all the templates used by the service. This is "more of the same" kind of work described previously, so I'll keep it short. The service instance I have been playing with is a very basic one, so it's covered by the single template we have already ported. But the service can do more. In particular, if I set a QoS parameter on the L3VPN service instance, the service will require the additional templates. Looking at the L3VPN service template directory, I find the following that relates to XE devices:

 

$ grep -rils urn:ios packages/l3vpn/templates/

packages/l3vpn/templates//l3vpn-acl.xml

packages/l3vpn/templates//l3vpn-ce.xml

packages/l3vpn/templates//l3vpn-pe.xml

packages/l3vpn/templates//l3vpn-qos-class.xml

packages/l3vpn/templates//l3vpn-qos-prio.xml

packages/l3vpn/templates//l3vpn-qos.xml

 

The l3vpn-ce.xml is what we already ported. The l3vpn-pe.xml is not relevant for the CE role. So four rather small templates remain to port. So let's repeat the work from the previous post. First, I will switch the device ce0 back to be a CLI device.

 

JLINDBLA-M-W0J2# con

Entering configuration mode terminal

JLINDBLA-M-W0J2(config)# devices device ce0 device-type cli ned-id cisco-ios

JLINDBLA-M-W0J2(config-device-ce0)# no port

JLINDBLA-M-W0J2(config-device-ce0)# comm

Commit complete.

JLINDBLA-M-W0J2(config-device-ce0)# top

JLINDBLA-M-W0J2(config)# devices sync-f

sync-result {

    device ce0

    result true

}

sync-result {

    device ce0-nc

    result true

}

sync-result {

    device ce1

    result true

 

Note that I synced ce0 (over CLI) as well as ce0-nc (over NETCONF) now, so we have a fresh baseline.

 

The service config I have running looks like this:

 

JLINDBLA-M-W0J2(config)# show full-configuration vpn

vpn l3vpn skoda

as-number 61000

endpoint newyork-office

  ce-device    ce3

  ce-interface GigabitEthernet0/2

  ip-network   10.1.0.0/24

  bandwidth    2000000

!

endpoint riodejaneiro-office

  ce-device    ce6

  ce-interface GigabitEthernet0/2

  ip-network   10.0.0.0/24

  bandwidth    10000000

!

endpoint tokyo-office

  ce-device    ce0

  ce-interface GigabitEthernet2

  ip-network   10.0.0.0/24

  bandwidth    10000000

!

!

 

I now need to extend this to also use some QoS service parameters.

 

JLINDBLA-M-W0J2(config)# vpn l3vpn skoda qos qos-policy GOLD                 

JLINDBLA-M-W0J2(config-l3vpn-skoda)# commit dry-run

cli {

    local-node {

        data  devices {

                  device ce0 {

                      config {

                          ios:ip {

                              access-list {

                                  extended {

             +                        ext-named-acl GLOBAL-call-signaling {

             +                            ext-access-list-rule "permit tcp any   any   range 5060 5061";

             +                        }

             +                        ext-named-acl GLOBAL-ssh {

             +                            ext-access-list-rule "permit tcp any   any   range 22 22";

             +                        }

             +                        ext-named-acl GLOBAL-voice {

             +                            ext-access-list-rule "permit udp any   any   range 16348 32767";

             +                        }

                                  }

                              }

                          }

             +            ios:class-map BUSINESS-CRITICAL {

             +                match-any;

             +                match {

             +                    access-group {

             +                        name GLOBAL-ssh;

             +                    }

             +                }

             +            }

             +            ios:class-map MISSION-CRITICAL {

             +                match-any;

             +                match {

             +                    access-group {

             +                        name GLOBAL-call-signaling;

             +                    }

             +                }

             +            }

...

 

JLINDBLA-M-W0J2(config-l3vpn-skoda)# commit

Commit complete.

JLINDBLA-M-W0J2(config-l3vpn-skoda)# top

 

The trick with showing the XML diff on the ce0-nc shadow device should be familiar by now.

 

JLINDBLA-M-W0J2(config)# devices device ce0-nc compare-config outformat xml

diff

<devices xmlns="http://tail-f.com/ns/ncs">

  <device>

    <name>ce0-nc</name>

    <config>

      <native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native">

        <ip>

          <access-list>

            <extended xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-acl">

              <name>GLOBAL-call-signaling</name>

              <access-list-seq-rule>

                <sequence>10</sequence>

                <ace-rule>

                  <action>permit</action>

                  <protocol>tcp</protocol>

                  <any/>

                  <dst-any/>

                  <dst-range1>5060</dst-range1>

                  <dst-range2>5061</dst-range2>

                </ace-rule>

              </access-list-seq-rule>

            </extended>

            <extended xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-acl">

              <name>GLOBAL-ssh</name>

              <access-list-seq-rule>

                <sequence>10</sequence>

                <ace-rule>

                  <action>permit</action>

                  <protocol>tcp</protocol>

                  <any/>

                  <dst-any/>

                  <dst-eq>22</dst-eq>

                </ace-rule>

              </access-list-seq-rule>

            </extended>

            <extended xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-acl">

              <name>GLOBAL-voice</name>

              <access-list-seq-rule>

                <sequence>10</sequence>

                <ace-rule>

                  <action>permit</action>

                  <protocol>udp</protocol>

                  <any/>

                  <dst-any/>

                  <dst-range1>16348</dst-range1>

                  <dst-range2>32767</dst-range2>

                </ace-rule>

              </access-list-seq-rule>

            </extended>

          </access-list>

        </ip>

        <policy>

          <class-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">

            <name>BUSINESS-CRITICAL</name>

            <prematch>match-any</prematch>

            <match>

              <access-group>

                <name>GLOBAL-ssh</name>

              </access-group>

            </match>

          </class-map>

          <class-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">

            <name>MISSION-CRITICAL</name>

            <prematch>match-any</prematch>

            <match>

              <access-group>

                <name>GLOBAL-call-signaling</name>

              </access-group>

            </match>

          </class-map>

          <class-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">

            <name>REALTIME</name>

            <prematch>match-any</prematch>

            <match>

              <access-group>

                <name>GLOBAL-voice</name>

              </access-group>

            </match>

          </class-map>

          <policy-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">

            <name>GOLD</name>

            <class>

              <name>BUSINESS-CRITICAL</name>

              <action-list>

                <action-type>bandwidth</action-type>

                <bandwidth>

                  <percent>20</percent>

                </bandwidth>

              </action-list>

              <action-list>

                <action-type>set</action-type>

                <set>

                  <ip>

                    <dscp>

                      <dscp-val>af21</dscp-val>

                    </dscp>

                  </ip>

                </set>

              </action-list>

            </class>

            <class>

              <name>MISSION-CRITICAL</name>

              <action-list>

                <action-type>bandwidth</action-type>

                <bandwidth>

                  <percent>25</percent>

                </bandwidth>

              </action-list>

              <action-list>

                <action-type>set</action-type>

                <set>

                  <ip>

                    <dscp>

                      <dscp-val>af31</dscp-val>

                    </dscp>

                  </ip>

                </set>

              </action-list>

            </class>

            <class>

              <name>REALTIME</name>

              <action-list>

                <action-type>priority</action-type>

                <priority>

                  <percent>20</percent>

                </priority>

              </action-list>

              <action-list>

                <action-type>set</action-type>

                <set>

                  <ip>

                    <dscp>

                      <dscp-val>ef</dscp-val>

                    </dscp>

                  </ip>

                </set>

              </action-list>

            </class>

          </policy-map>

          <policy-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">

            <name>skoda</name>

            <class>

              <name>class-default</name>

              <action-list>

                <action-type>service-policy</action-type>

                <service-policy>GOLD</service-policy>

              </action-list>

            </class>

          </policy-map>

        </policy>

        <router>

          <bgp xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-bgp">

            <id>61000</id>

            <neighbor>

              <id>192.168.1.2</id>

              <remote-as>100</remote-as>

            </neighbor>

            <address-family>

              <no-vrf>

                <ipv4>

                  <af-name>unicast</af-name>

                  <neighbor>

                    <id>192.168.1.2</id>

                    <activate/>

                  </neighbor>

                </ipv4>

              </no-vrf>

            </address-family>

          </bgp>

        </router>

      </native>

    </config>

  </device>

</devices>

 

Now, matching this towards the templates in the service package, like we did previously, I end up with new templates as follows:

 

l3vpn-qos.xml:

 

        <native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" tags="merge">

          <policy>

            <policy-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">

              <name>{/name}</name>

              <class>

                <name>class-default</name>

                <action-list>

                  <action-type>service-policy</action-type>

                  <service-policy>{$POLICY_NAME}</service-policy>

                </action-list>

              </class>

            </policy-map>

          </policy>

        </native>

 

l3vpn-qos-prio.xml:

 

        <native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" tags="merge">

          <policy>

            <policy-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">

              <name>{$POLICY_NAME}</name>

              <class>

                <name>{$CLASS_NAME}</name>

                <action-list>

                  <action-type>priority</action-type>

                  <priority>

                    <percent>{$CLASS_BW}</percent>

                  </priority>

                </action-list>

                <action-list>

                  <action-type>set</action-type>

                  <set>

                    <ip>

                      <dscp>

                        <dscp-val>{$CLASS_DSCP}</dscp-val>

                      </dscp>

                    </ip>

                  </set>

                </action-list>

              </class>

            </policy-map>

          </policy>

        </native>

 

l3vpn-qos-class.xml:

 

        <native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" tags="merge">

          <policy>

            <class-map xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-policy">

              <name>{$CLASS_NAME}</name>

              <prematch>match-any</prematch>

              <match>

                <access-group>

                  <name>{$MATCH_ENTRY}</name>

                </access-group>

              </match>

            </class-map>

          </policy>

        </native>

 

l3vpn-acl.xml:

 

        <native xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-native" tags="merge">

          <ip>

            <access-list>

              <extended xmlns="http://cisco.com/ns/yang/Cisco-IOS-XE-acl">

                <name>{$ACL_NAME}</name>

                <access-list-seq-rule>

                  <sequence>10</sequence>

                  <ace-rule>

                    <action>permit</action>

                    <protocol>{$PROTOCOL}</protocol>

                    <any/><!-- FIXME: {$SOURCE_IP_ADR} {$SOURCE_WMASK} -->

                    <dst-any/><!-- FIXME: {$DEST_IP_ADR} {$DEST_WMASK} -->

                    <dst-range1 when="{boolean($PORT_START) and boolean($PORT_END) and ($PORT_START!=$PORT_END)}">{$PORT_START}}</dst-range1>

                    <dst-range2 when="{boolean($PORT_START) and boolean($PORT_END) and ($PORT_START!=$PORT_END)}">{$PORT_END}</dst-range2>

                    <dst-eq when="{boolean($PORT_START) and (not($PORT_END) or ($PORT_END=$PORT_START))}">{$PORT_START}</dst-eq>

                  </ace-rule>

                </access-list-seq-rule>

              </extended>

            </access-list>

          </ip>

        </native>

 

Here, in the ACL template, a few quirks are worth discussing. In the old CLI-based template, the ACL template was about constructing a string something like this: "premit tcp any any range 5060 5061". In the YANG model used over the NETCONF interface, this line has been split up into individual leafs (great!). When it comes to the port/port-range, I have to use two different constructs. dst-range1 .. dst-range2 if there is an actual range, and dst-eq if it's a single port. The application code could be updated to apply different templates for each case, but I'm good at XPATH, so I can construct some when expressions to ensure the right alternative is used depending on the service instance input data. This way I don't have to change any of the service code.

 

Also, the CLI-based template was made to inject the source and destination IP addresses in there. I didn't figure out what I needed to do in the service input data to make them actually be used, so I just commented out those XPATH variables with a FIXME. If anyone knows when they would be used, I could add that later.

 

So with these new templates, let's see what happens:

 

JLINDBLA-M-W0J2# con           

Entering configuration mode terminal

JLINDBLA-M-W0J2(config)# vpn l3vpn skoda qos qos-policy GOLD

JLINDBLA-M-W0J2(config-l3vpn-skoda)# commit dry-run                    

cli {

    local-node {

        data  devices {

                  device ce0 {

                      config {

                          ios-native:native {

                              ip {

                                  access-list {

             +                        extended GLOBAL-call-signaling {

             +                            access-list-seq-rule 10 {

             +                                ace-rule {

             +                                    action permit;

             +                                    protocol tcp;

             +                                    any;

             +                                    dst-any;

             +                                    dst-range1 5060;

             +                                    dst-range2 5061;

             +                                }

             +                            }

             +                        }

             +                        extended GLOBAL-ssh {

             +                            access-list-seq-rule 10 {

             +                                ace-rule {

             +                                    action permit;

             +                                    protocol tcp;

             +                                    any;

             +                                    dst-any;

             +                                    dst-eq 22;

             +                                }

             +                            }

             +                        }

             +                        extended GLOBAL-voice {

             +                            access-list-seq-rule 10 {

             +                                ace-rule {

             +                                    action permit;

             +                                    protocol udp;

             +                                    any;

             +                                    dst-any;

             +                                    dst-range1 16348;

             +                                    dst-range2 32767;

             +                                }

             +                            }

             +                        }

                                  }

                              }

                              policy {

             +                    class-map BUSINESS-CRITICAL {

             +                        prematch match-any;

             +                        match {

             +                            access-group {

             +                                name [ GLOBAL-ssh ];

             +                            }

             +                        }

             +                    }

             +                    class-map MISSION-CRITICAL {

             +                        prematch match-any;

             +                        match {

             +                            access-group {

             +                                name [ GLOBAL-call-signaling ];

             +                            }

             +                        }

             +                    }

             +                    class-map REALTIME {

             +                        prematch match-any;

             +                        match {

             +                            access-group {

             +                                name [ GLOBAL-voice ];

             +                            }

             +                        }

             +                    }

             +                    policy-map GOLD {

             +                        class REALTIME {

             +                            action-list priority {

             +                                priority {

             +                                    percent 20;

             +                                }

             +                            }

             +                            action-list set {

             +                                set {

             +                                    ip {

             +                                        dscp {

             +                                            dscp-val ef;

             +                                        }

             +                                    }

             +                                }

             +                            }

             +                        }

             +                    }

                                  policy-map skoda {

                                      class class-default {

             +                            # after action-list shape

             +                            action-list service-policy {

             +                                service-policy GOLD;

             +                            }

                                      }

                                  }

                              }

                          }

                      }

                  }

                  device ce3 {

                      config {

                          ios:ip {

...

 

Looks good. Let's try the real test then:

 

JLINDBLA-M-W0J2(config-l3vpn-skoda)# comm

Commit complete.

JLINDBLA-M-W0J2(config-l3vpn-skoda)# top

JLINDBLA-M-W0J2(config)# devices device ce0 compare-config

JLINDBLA-M-W0J2(config)# vpn l3vpn skoda un-deploy       

JLINDBLA-M-W0J2(config)# devices device ce0 compare-config

JLINDBLA-M-W0J2(config)# vpn l3vpn skoda re-deploy       

JLINDBLA-M-W0J2(config)# devices device ce0 compare-config

JLINDBLA-M-W0J2(config)#

 

Yay! We're able to both apply and remove the service instance, and after each change there is zero diff. NSO and the device keeps in sync (with some workarounds in the service, sure).