YDK Performance issue

[magedahmed]

I am facing performance issue to add 2000 interfaces information under YANG models

- yang model "from ydk.models.cisco_ios_xr import Cisco_IOS_XR_infra_statsd_oper" take about 30 seconds

- yang model 'from ydk.models.juniper_show_interfaces import show_interfaces' take about 200 seconds

is there any way to improve performance issue?

below the used code with Cisco YANG model

def interfaces_fun(self):
    fsm_results, re_header_dic = self.parse_text_fsm()
    self.interfaces = Cisco_IOS_XR_infra_statsd_oper.InfraStatistics.Interfaces()
    one_interface_info = Cisco_IOS_XR_infra_statsd_oper.InfraStatistics.Interfaces.Interface()
    for row in fsm_results:
        interface_name = row[re_header_dic['INTERFACE']]
        one_interface_info.interface_name = interface_name
        one_interface_info.data_rate.input_data_rate = int(row[re_header_dic['INPUT_RATE_BITS']] or 0)
        one_interface_info.data_rate.output_data_rate = int(row[re_header_dic['OUTPUT_RATE_BITS']] or 0)
        
        self.interfaces.interface.append(one_interface_info)

    return self.interfaces

@ygorelik 

[saalvare]

What exactly is taking 200s? The import statement? The execution of the RPC? You may want to capture debug logs and share them.

import logging

logger = logging.getLogger("ydk")
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler()
formatter = logging.Formatter(("%(asctime)s - %(name)s - "
                                      "%(levelname)s - %(message)s"))
handler.setFormatter(formatter)
logger.addHandler(handler)

[magedahmed]

I am converting some data from text to json using textfsm and YDK

it take 30 second for 2000 interface when I am using "Cisco-IOS-XR-infra-statsd-oper:interfaces" yang model and it take 200 second with another customized YANG Model.

I am trying to minimize the time to 1 or 2 seconds

below sample for two interfaces

attached debugging file or below link

https://drive.google.com/open?id=188N1bbozpIJO9n4m4uAf5rrhatmwfxDU

"Cisco-IOS-XR-infra-statsd-oper:interfaces": {
"interface": [
{
"interface-name": "BVI30",
"data-rate": {
"input-data-rate": "8000",
"output-data-rate": "5000"
}
},
{
"interface-name": "BVI2000",
"data-rate": {
"input-data-rate": "0",
"output-data-rate": "0"
}
},

below function that take 200 seconds to get the json output

def interfaces_fun(self):
    fsm_results, re_header_dic = self.parse_text_fsm()
    end = time.time()
    print("Total time Before Loop: {}".format(end - start))

    self.interfaces = show_interfaces.InterfaceInformation()

    for row in fsm_results:
        interface_name = row[re_header_dic['INTERFACE']]
        one_interface_info = self.check_phy_logical(interface_name)
        one_interface_info.name = interface_name
        self.interfaces.physical_interface.append(one_interface_info)

    return self.interfaces

below sample output for the code that take 200 second to generate the json output

"show-interfaces:interface-information": {
"logical-interface": [
{
"name": "BVI30"
},
{
"name": "BVI2000"
},
{
"name": "BVI2077"
},

[abhirame]

@magedahmed 

Based on my understanding, the issue seems to be that the import statement is taking too long. Correct?

Can you please respond with the output of the below from command line to verify the issue?

time python -c 'from ydk.models.cisco_ios_xr import Cisco_IOS_XR_infra_statsd_oper'

time python -c 'from ydk.models.juniper_show_interfaces import show_interfaces'

If the import statement is taking long, it may mean you have to increase your RAM. The Cisco-IOS-XR-infra-statsd-oper.yang model is not that big.

[magedahmed]

@abhirame

below the required output, Could you please advise if it is related to RAM or not?

auto@yang-vm:~$ time python3 -c 'from ydk.models.cisco_ios_xr import Cisco_IOS_XR_infra_statsd_oper'

real 0m7.505s
user 0m0.364s
sys 0m0.795s
auto@yang-vm:~$

## The below about after I changed the yang file and it is very small now
auto@yang-vm:~$ time python3 -c 'from ydk.models.juniper_show_interfaces import show_interfaces'

real 0m1.159s
user 0m0.349s
sys 0m0.033s