topic Re: Problems using YDK with Huawei YANG files in Tools

Problems using YDK with Huawei YANG files

geordish1 — Wed, 07 Mar 2018 21:28:01 GMT

Hi,

I'm trying to use ydk to configure Huawei VRP devices using netconf.

So far, I've managed to generate the ydk for the Huawei YANG files (located here: https://github.com/Huawei/yang), and have started to play with the API.

I've got a simple bit of code as so:

from ydk.services import CRUDService
from ydk.providers import NetconfServiceProvider
from ydk.models.huawei import huawei_ifm

provider = NetconfServiceProvider(address="cor1.lond2.lab",
                                      port=22,
                                      username="username",
                                      password="password",
                                      protocol="ssh")


ifm = huawei_ifm.Ifm.Interfaces.Interface()
ifm.ifname = 'GigabitEthernet1/1/1.200'
ifm.ifparentifname = 'GigabitEthernet1/1/1'
ifm.ifmtu = 9600
ifm.ifdescr = "This is a test description"

crud = CRUDService()
interface = crud.create(provider, ifm)

From the debugs, the netconf session connects, so that part all looks good. The issue appears to be with the generating of the API call.

Traceback (most recent call last):
  File "test.py", line 19, in <module>
    interface = crud.create(provider, ifm)
  File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 112, in helper
    return func(self, provider, entity, *args, **kwargs)
  File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/services/crud_service.py", line 30, in create
    return self._crud.create(provider, entity)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 82, in handle_runtime_error
    _raise(_exc)
  File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 56, in _raise
    raise exc
ydk.errors.YPYInvalidArgumentError:  Path is invalid: huawei-ifm:ifm/interfaces/interface[ifName='GigabitEthernet1/1/1.200']

The path according to the YANG file is valid however:

yang -f tree --tree-path="ifm/interfaces" huawei-ifm.yang | head -n 6

module: huawei-ifm

+--rw ifm

+--rw interfaces

+--rw interface* [ifName]

+--rw ifName pub-type:ifName

+--ro ifIndex? uint32

Using the CodecService this does appear to generate something that looks correct however. If I alter my code, I get this:

Code:

from ydk.services import CodecService
from ydk.providers import CodecServiceProvider
from ydk.models.huawei import huawei_ifm

provider = CodecServiceProvider(type='xml')
codec = CodecService()

ifm = huawei_ifm.Ifm.Interfaces.Interface()

ifm.ifname = 'GigabitEthernet1/1/1.200'
ifm.ifparentifname = 'GigabitEthernet1/1/1'
ifm.ifmtu = 9600
ifm.ifdescr = "This is a test description"

print (codec.encode(provider, ifm))

Output:

<interface xmlns="http://www.huawei.com/netconf/vrp/huawei-ifm">
  <ifName>GigabitEthernet1/1/1.200</ifName>
  <ifParentIfName>GigabitEthernet1/1/1</ifParentIfName>
  <ifDescr>This is a test description</ifDescr>
  <ifMtu>9600</ifMtu>
</interface>

Any ideas or clues would be greatly appreciated!

Dave

Re: Problems using YDK with Huawei YANG files

abhirame — Thu, 08 Mar 2018 21:00:28 GMT

Hi Dave,

Can you please try the below? You can try passing in the top level object

from ydk.services import CRUDService  
from ydk.providers import NetconfServiceProvider  
from ydk.models.huawei import huawei_ifm  
  
provider = NetconfServiceProvider(address="cor1.lond2.lab",  
                                      port=22,  
                                      username="username",  
                                      password="password",  
                                      protocol="ssh")  
  

ifm_obj =   huawei_ifm.Ifm()

ifm = huawei_ifm.Ifm.Interfaces.Interface()  
ifm.ifname = 'GigabitEthernet1/1/1.200'  
ifm.ifparentifname = 'GigabitEthernet1/1/1'  
ifm.ifmtu = 9600  
ifm.ifdescr = "This is a test description"  

ifm_obj.interfaces.interface.append(ifm)
  
crud = CRUDService()  
interface = crud.create(provider, ifm_obj)

See the below open issue for why this is needed:

https://github.com/CiscoDevNet/ydk-gen/issues/603

Thanks,

Abhi

Re: Problems using YDK with Huawei YANG files

geordish1 — Thu, 08 Mar 2018 21:17:48 GMT

Hi Abhi,

Thanks for your response.

I gave that a go, but no luck. I now get the following error.

Traceback (most recent call last):
  File "test2.py", line 23, in <module>
    interface = crud.create(provider, ifm_obj)
  File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 112, in helper
    return func(self, provider, entity, *args, **kwargs)
  File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/services/crud_service.py", line 30, in create
    return self._crud.create(provider, entity)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 82, in handle_runtime_error
    _raise(_exc)
  File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 56, in _raise
    raise exc
ydk.errors.YPYInvalidArgumentError:  Path is invalid: huawei-ifm:ifm

Using the Codec Service to print out the contents of ifm_obj does look better however:

<ifm xmlns="http://www.huawei.com/netconf/vrp/huawei-ifm">
  <interfaces>
    <interface>
      <ifName>GigabitEthernet1/1/1.200</ifName>
      <ifParentIfName>GigabitEthernet1/1/1</ifParentIfName>
      <ifDescr>This is a test description</ifDescr>
      <ifMtu>9600</ifMtu>
    </interface>
  </interfaces>
</ifm>

This is the same XML that I use when creating the netconf requests manually.

Regards,

Dave

Re: Problems using YDK with Huawei YANG files

geordish1 — Thu, 08 Mar 2018 21:44:30 GMT

Further to this, I've added some logging, and am seeing the following:

Path where models are to be downloaded: /home/dave/.ydk/cor1.lond2.lab:22

Connected to cor1.lond2.lab on port 22 using ssh with timeout of -1

Executing CRUD create operation

Data is invalid according to the yang model. Error details: Invalid keyword "&#13".

Data is invalid according to the yang model. Error details: Module parsing failed.

Data is invalid according to the yang model. Error details: Invalid keyword "&#13".

Data is invalid according to the yang model. Error details: Module parsing failed.

Data is invalid according to the yang model. Error details: Importing "huawei-ifm" module into "huawei-ifm-deviations-OC-NE-X3" failed.

Data is invalid according to the yang model. Error details: Module "huawei-ifm-deviations-OC-NE-X3" parsing failed.

Data is invalid according to the yang model. Error details: Invalid keyword "&#13".

Data is invalid according to the yang model. Error details: Module parsing failed.

Data is invalid according to the yang model. Error details: Invalid keyword "&#13".

Data is invalid according to the yang model. Error details: Module parsing failed.

Data is invalid according to the yang model. Error details: Importing "huawei-ifm" module into "huawei-ifm-deviations-OC-NE-X3" failed.

Data is invalid according to the yang model. Error details: Module "huawei-ifm-deviations-OC-NE-X3" parsing failed.

Data is invalid according to the yang model. Error details: Module not found. Path: '/huawei-ifm'

Path 'huawei-ifm:ifm' is invalid

Traceback (most recent call last):

File "test2.py", line 28, in <module>

interface = crud.create(provider, ifm_obj)

File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 112, in helper

return func(self, provider, entity, *args, **kwargs)

File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/services/crud_service.py", line 30, in create

return self._crud.create(provider, entity)

File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__

self.gen.throw(type, value, traceback)

File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 82, in handle_runtime_error

_raise(_exc)

File "/home/dave/.virtualenvs/py2/local/lib/python2.7/site-packages/ydk/errors/error_handler.py", line 56, in _raise

raise exc

ydk.errors.YPYInvalidArgumentError: Path is invalid: huawei-ifm:ifm

Disconnected from device

Looking at the files that are in the folder /home/dave/.ydk/cor1.lond2:22, I'm seeing a lot of junk:

/*&#13;
Copyright (C) 2013-2017 Huawei Technologies Co., Ltd. All rights reserved.&#13;
*/&#13;
module huawei-ifm {&#13;
  namespace "http://www.huawei.com/netconf/vrp/huawei-ifm";&#13;
  prefix ifm;&#13;
  import huawei-pub-type {&#13;
    prefix pub-type;&#13;
  }&#13;
  import huawei-rsa {&#13;
    prefix rsa;&#13;
  }&#13;
  import ietf-inet-types {&#13;
    prefix inet;&#13;
  }&#13;
  import huawei-extension {&#13;
    prefix ext;&#13;
  }&#13;

Where are these models being obtained from? Is it from the device itself?

Regards,

Dave

Re: Problems using YDK with Huawei YANG files

geordish1 — Fri, 09 Mar 2018 15:20:52 GMT

I've done more digging. The yang models it is complaining about are indeed being sent by the device.

I believe this could be a YDK bug. When debugging on the device I am attempting to configure, it shows the module being sent looking like:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply message-id="1" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <data xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring">/*&#13;
Copyright (C) 2013-2017 Huawei Technologies Co., Ltd. All rights reserved.&#13;
*/&#13;
submodule huawei-rsa-type {&#13;
  belongs-to huawei-rsa {&#13;
    prefix rsa;&#13;
  }&#13;
  &#13;
  organization&#13;
    &quot;Huawei Technologies Co.,Ltd.&quot;;&#13;
  contact&#13;
    &quot;Huawei Industrial

As this is XML, I think the character encoding is probably OK. I notice that some of them are correctly being decoded, such as &qout; for quotes when it is stored in the cache directory under ~/.ydk/device/

The codes I've encountered so far which are not being decoded are:

I tried to investigate into the ydk sdk code, but I have no clue as to where any decoding may be done. Can anyone confirm if this encoding should be there, so I can rase an issue with the correct people.

I fixed up the local cache of the downloaded YANG files, and its now generating an RPC call:

<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<edit-config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <target>
    <candidate/>
  </target>
  <error-option>rollback-on-error</error-option>
  <config><ifm xmlns="http://www.huawei.com/netconf/vrp/huawei-ifm" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" nc:operation="merge">
  <interfaces>
    <interface>
      <ifName>GigabitEthernet0/2/6.200</ifName>
      <ifParentIfName>GigabitEthernet0/2/6</ifParentIfName>
      <ifDescr>This is a test description</ifDescr>
      <ifMtu>9600</ifMtu>
    </interface>
  </interfaces>
</ifm>
</config>
</edit-config>
</rpc>

The response I'm getting is the following:

<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="1">
  <rpc-error>
    <error-type>application</error-type>
    <error-tag>operation-not-supported</error-tag>
    <error-severity>error</error-severity>
    <error-message>This operation is not supported.</error-message>
    <error-info>
      <bad-attribute>operation</bad-attribute>
      <bad-element>ifm</bad-element>
    </error-info>
  </rpc-error>
</rpc-reply>

Further investigation shows that the kit I'm using doesn't support merge - it only supports create. The documentation I'm reading says that ydk doesn't support create, only merge. Is this right?

Dave

Re: Problems using YDK with Huawei YANG files

abhirame — Fri, 09 Mar 2018 23:26:20 GMT

The issue with weird characters was fixed recently. The fix will be included as part of the next release of ydk (0.7.1)

https://github.com/CiscoDevNet/ydk-gen/issues/692

Re: Problems using YDK with Huawei YANG files

geordish1 — Sun, 11 Mar 2018 23:18:02 GMT

Thanks for your response. I've submitted a PR to fix the  also.

Do you have any ideas about the second part of my issue?

Dave

Re: Problems using YDK with Huawei YANG files

abhirame — Mon, 12 Mar 2018 02:55:30 GMT

Sorry I missed that. YDK does support create. Not sure where you saw that YDK only supports merge.

Please refer to the example here and use 'YFIlter.create' instead of 'YFilter.replace':

http://ydk.cisco.com/py/docs/guides/crud_guide.html#creating-and-replacing-a-configuration

Re: Problems using YDK with Huawei YANG files

geordish1 — Mon, 12 Mar 2018 10:36:12 GMT

Thanks for your help so far, I feel I'm getting very close.

This is what I have so far:

from ydk.filters import YFilter
from ydk.services import CRUDService
from ydk.providers import NetconfServiceProvider
from ydk.models.huawei import huawei_ifm

# Logging
import logging
log = logging.getLogger('ydk')
log.setLevel(logging.INFO)
handler = logging.StreamHandler()
log.addHandler(handler)

provider = NetconfServiceProvider(address="cor1.lond2.lab",
                                      port=22,
                                      username="user",
                                      password="pass",
                                      protocol="ssh")

ifm_obj =  huawei_ifm.Ifm()

ifm = huawei_ifm.Ifm.Interfaces.Interface()
ifm.ifname = 'GigabitEthernet0/2/6.200'
ifm.ifparentifname = 'GigabitEthernet0/2/6'
ifm.ifmtu = 9600
ifm.ifdescr = "This is a test description"

ifm.yfilter = YFilter.create

ifm_obj.interfaces.interface.append(ifm)
ifm_obj.yfilter = YFilter.not_set

crud = CRUDService()
interface = crud.create(provider, ifm_obj)

This produces the following:

<rpc xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<edit-config xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <target>
    <candidate/>
  </target>
  <error-option>rollback-on-error</error-option>
  <config><ifm xmlns="http://www.huawei.com/netconf/vrp/huawei-ifm" xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0" nc:operation="merge">
  <interfaces>
    <interface nc:operation="create">
      <ifName>GigabitEthernet0/2/6.200</ifName>
      <ifParentIfName>GigabitEthernet0/2/6</ifParentIfName>
      <ifDescr>This is a test description</ifDescr>
      <ifMtu>9600</ifMtu>
    </interface>
  </interfaces>
</ifm>
</config>
</edit-config>
</rpc>

That merge operation on the ifm object needs to be removed. Setting the YFilter on the ifm object seems to do nothing. Am I doing something wrong again?

Dave

Re: Problems using YDK with Huawei YANG files

abhirame — Mon, 12 Mar 2018 14:56:47 GMT

CRUD Service create always uses the merge operation. You may want to use the NetconfService (http://ydk.cisco.com/py/docs/api/services/netconf_service.html) which lets you define custom operations.

interface = netconf_service.edit_config(provider, DataStore.running, ifm_obj)

Re: Problems using YDK with Huawei YANG files

geordish1 — Mon, 12 Mar 2018 17:05:10 GMT

Awesome, that was the last push I needed! Using the netconf service works well.

For completion, my full test script is as follows.

from ydk.filters import YFilter
from ydk.services import NetconfService, Datastore
from ydk.providers import NetconfServiceProvider
from ydk.models.huawei import huawei_ifm

# Logging
import logging
log = logging.getLogger('ydk')
log.setLevel(logging.INFO)
handler = logging.StreamHandler()
log.addHandler(handler)

provider = NetconfServiceProvider(address="cor1.lond2.lab",
                                      port=22,
                                      username="user",
                                      password="pass",
                                      protocol="ssh")
netconf = NetconfService()

ifm_obj =   huawei_ifm.Ifm()
ifm = huawei_ifm.Ifm.Interfaces.Interface()
ifm.ifname = 'GigabitEthernet1/1/1.200'
ifm.ifparentifname = 'GigabitEthernet1/1/1'
ifm.ifmtu = 9600
ifm.ifdescr = "This is a test description"
ifm.yfilter = YFilter.create

ifm_obj.interfaces.interface.append(ifm)
interface = netconf.edit_config(provider, Datastore.running, ifm_obj)

Regards,

Dave

Re: Problems using YDK with Huawei YANG files

abhirame — Mon, 12 Mar 2018 18:17:01 GMT

Great! Glad it worked