Couldn't fetch schema node of an augmented object in another YANG file when encoding in XML

[tqhuy]

Hello,

I'm having an error when trying to encode an Entity in XML with ydk 0.8.5.

For example, I have a "connectivity-service" grouping with child "end-point" described here tapi-connectivity. In another yang model tapi-photonic, the "end-point" is augmented with some specifications.

Everything works fine when I assign a value for one of the specifications of the "end-point". For example:

...
end_point = Connectivity.EndPoint()
end_point.specs.power = 5.5

However, since the specs is an augmentation of end-point in another yang file, I got an error when I tried to encode to XML.

codec = CodecService()
provider = CodecServiceProvider(type="xml")
end_point_xml = codec.encode(provider,end_point)

ydk.errors.YCoreError:  Could not fetch schema node: specs

From my understanding, the "_get_data_node_from_entity" method cannot find the schema for "specs". However, in the generated Connectivity python class, I can see the "specs" attributes as well as its children.

Could you please help me with this issue? Thank you very much for your support.

Best regards,

Huy

[yangorelik]

The 'end-point' is not a top level container, but rather a list element under long hierarchy. If you want just to get XML presentation of that object, you need to add `subtree=True` parameter like this:

end_point_xml = codec.encode(provider,end_point,subtree=True)

[tqhuy]

Hi Yan,

Thank you for your response.

It's true that I need to add subtree=Tree for the case of end_point.

However, I still get the error 

RuntimeError: YServiceError: Could not find node tapi-photonic:specs

If I call any grouping, which is defined in the same Yang file of the root node (Connectivity), I can encode normally. For instance (I re-write my code snippet example), the grouping request-capacity is defined in the same tapi-connectivity.yang

connectivity = Connectivity()
connectivity.request_capacity.total_capacity = 100 # If I encode connectivity at this point, there is no error
connectivity.end_point.specs.power = 5.5     # If I encode connectivity at this point with subtree=True, it shows the above error 

Thank you for your help.

Best,

Huy

[yangorelik]

For further troubleshooting please send me the bundle package (tar.gz file) and the script file.Also it would be helpful for you to get hierarchy tree of your root model:
pyang -f tree your-module.yang

[tqhuy]

Hi Yan,

I put everything in the attached .zip file, since the page does not allow to upload .py and .tar.gz file.

The tree hierarchy of related YANG models can be found here:

tapi-common@2020-04-23.tree 

tapi-connectivity@2020-06-16.tree 

tapi-photonic-media@2020-06-16.tree 

For simplicity, you can see the snapshot of the tree in yangsuite

 

Thank you for your support in advance.

Best,

Huy.

[yangorelik · Accepted solution]

The issue here is that CodecService uses XmlSubtreeCodec to get XML subtree of non-top level entity. In this case neither CodecService nor XmlSubtreeCodec loads all the YANG modules into the root schema. The last is needed for traversing the model. Because augmented modules are not explicitly listed, they got left not loaded to the root schema.

To overcome this issue I would suggest the following:

1. Initialize root schema by connecting to device with NetconfSession or NetconfServiceProvider. When YDK connects to device, it loads all listed in capabilities modules to the root schema, which will include the augmented modules.
2. Use XmlSubtreeCodec directly to get XML subtree.

Example:

repo = Repository('path-to-YANG-model-directory')
session = NetconfSession(repo=repo, hostname='***', port=830, username='admin', password='Cisco123')
root_schema = session.get_root_schema()

xml_codec = XmlSubtreeService()
end_point_xml = xml_codec.encode(end_point, root_schema)

There are also other ways to initialize root schema, but they are more complicated and require manual definition of all the Capabilities and enabled if-features. Let me know if you decide to go that route.

[tqhuy]

Hi Yan,

Thanks to your suggestion, I am able to encode by populating the Capabilities of all TAPI YANG models using the Moco Runner Rest Server in ydk-gen/test/db.json. I leave the code snippet here in case someone may need it.

# First, add YANG modules to the uri "/restconf/data/ietf-restconf-monitoring:restconf-state/capabilities" in ydk-gen/test/db.json , then start the Moco Runner rest server

context1 = ydk.models.tapi.tapi_common.Context()
yang_path = _get_yang_path(context1)
repo = Repository(yang_path)
session = RestconfSession(repo=repo, address='127.0.0.1', port=12306, username='admin', password='Cisco123')
root_schema = session.get_root_schema()
xml_codec = XmlSubtreeCodec()
context1_xml = xml_codec.encode(context1, root_schema)
print("context1_xml")

Nevertheless, I'm curious why you don't let CodecServiceProvider load all augmented modules to the root schema when using the Xml CodecServiceProvider._initialize_root_schema by the following code; but, the Restconf/Netconf session can.

from ydk.services.codec_service import CodecService, _get_yang_path
from ydk.providers import CodecServiceProvider
from ydk.entity_utils import XmlSubtreeCodec,_get_bundle_name
from ydk.path import Repository

context1 = ydk.models.tapi.tapi_common.Context()
codec = CodecService()
xml_provider = CodecServiceProvider(type="xml")
yang_path = _get_yang_path(context1)
repo = Repository(yang_path)
bundle_name = _get_bundle_name(context1)
xml_provider._initialize_root_schema(bundle_name, repo)
root_schema = xml_provider.get_root_schema(bundle_name)
context1_xml = xml_codec.encode(context1, root_schema) #Error here
print(context1_xml)

After establishing the session, are Capabilities downloaded locally in some directory? In my case, I don't have the ~/.ydk directory.

Thank you.

Best,

Huy

[yangorelik]

The last example is too complicated. The key here is that context1 is a top level entity. Therefore it should be simple, like this (with no errors):

from ydk.services.codec_service import CodecService
from ydk.providers import CodecServiceProvider

context1 = ydk.models.tapi.tapi_common.Context()
codec_service = CodecService()
xml_provider = CodecServiceProvider(type="xml")
context1_xml = codec_service.encode(xml_provider, context1)
print(context1_xml)

The CodecService internally will initialize the root schema and use it for the job.

In initial example you tried to encode non-top level entity end_point, which created the whole issue. In general, when building a model object you should always start from top level entity (container/list) and specify all the keys for the list elements going down the tree. That will assure correct building of the XML/JSON strings for all the entities, top and non-top.

Regarding capabilities. When service provider connects to device using Restconf/Netconf/gNMI, it always gets device capabilities. The capabilities are stored in memory. The directory ~/.ydk is used to store YANG files on-demand downloaded from device by Netconf protocol. This way YDK builds YANG module repository, when it is not defined by the user. In your case, the YDK uses model bundle as a repository, hence the ~/.ydk directory is empty.

[tqhuy]

Many thanks for the clarification, Yan. 

Best,

Huy