issue in genie parser for command "show ip bgp vpnv4 vrf VRF-NAME"

[thibaultm]

Hello, I discovered genie parser and it's awesome.
I installed pyats[library] + genie + pyats-genie-command-parse 1.3.9 on ubuntu 20.04
I use it on .txt files as explained at the url :
https://community.cisco.com/t5/controllers/use-genie-parser-on-text-files/m-p/4588903

I run it on pycharm so I can correct my mistakes easily.
I have seen the available list of parser for os at the url :
https://pubhub.devnetcloud.com/media/genie-feature-browser/docs/#/parsers

My installation works very well for the command :
show ip route vrf VRF-NAME

But even though the command "show ip bgp vpnv4 vrf VRF-NAME" is listed as supported at the above url of supported parsers (n.b. for os = 'iosxe') it does not work.

I keep having the following error :
pyats_genie_command_parse.py", line 168, in __parse
raise ModuleNotFoundError(f'Could not find module_name for command {show_command} '
ModuleNotFoundError: Could not find module_name for command show ip bgp vpnv4 vrf MPLS for nos iosxe from genie: local variable 'original_address_family' referenced before assignment.

In order to provide complete technical context :
I run pyats-genie-command-parse 1.3.9 and I upgraded genie pyats library to 24.2 before running the script.

The full script I run in pycharm is :
from pyats_genie_command_parse import GenieCommandParse
import json
#
parse_obj = GenieCommandParse(nos='iosxe')
address_family = "vpnv4"
vrf = "MPLS"
test = f'show ip bgp {address_family} vrf {vrf}'
print(f'show ip bgp {address_family} vrf {vrf}')
data = parse_obj.parse_file(show_command=f'show ip bgp {address_family} vrf {vrf}', file_name_and_path='./PE_R1_show_ip_bgp_vpnv4_vrf_MPLS.txt')
with open("json_line.txt", "a") as j:
print(data, file=j)
with open("json_indented.txt", "a") as i:
print('\n', json.dumps(data, indent=2), file=i)

and the file PE_R1_show_ip_bgp_vpnv4_vrf_MPLS.txt is :

PE-R1#show ip bgp vpnv4 vrf MPLS
BGP table version is 19, local router ID is 1.1.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path, L long-lived-stale,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf MPLS)
*>i 4.4.4.4/32 3.3.3.3 20 100 0 ?
*> 5.5.5.5/32 10.0.15.5 0 0 55 ?
r> 10.0.15.0/24 10.0.15.5 0 0 55 ?
*>i 10.0.34.0/24 3.3.3.3 0 100 0 ?
*>i 172.44.0.0/24 3.3.3.3 20 100 0 ?
*>i 172.44.10.0/24 3.3.3.3 20 100 0 ?
*>i 172.44.20.0/24 3.3.3.3 20 100 0 ?
*>i 172.44.30.0/24 3.3.3.3 20 100 0 ?
*> 172.55.0.0/24 10.0.15.5 0 0 55 ?
*> 172.55.10.0/24 10.0.15.5 0 0 55 ?
*> 172.55.20.0/24 10.0.15.5 0 0 55 ?
*> 172.55.30.0/24 10.0.15.5 0 0 55 ?
PE-R1#

The full result I  get when I run the script in pycharm is :

/home/thibault/PycharmProjects/test-genie-parser/venv/bin/python /home/thibault/PycharmProjects/test-g-parser/main.py
show ip bgp vpnv4 vrf MPLS
Traceback (most recent call last):
File "/home/thibault/PycharmProjects/test-genie-parser/venv/lib/python3.8/site-packages/pyats_genie_command_parse/pyats_genie_command_parse.py", line 165, in __parse
return found_parser(device=mock_device).parse()
File "src/genie/metaparser/_metaparser.py", line 308, in genie.metaparser._metaparser.MetaParser.parse
File "/home/thibault/PycharmProjects/test-genie-parser/venv/lib/python3.8/site-packages/genie/libs/parser/iosxe/show_bgp.py", line 766, in cli
return super().cli(output=show_output, vrf=vrf,
File "/home/thibault/PycharmProjects/test-genie-parser/venv/lib/python3.8/site-packages/genie/libs/parser/iosxe/show_bgp.py", line 529, in cli
new_address_family = original_address_family + ' RD ' + route_distinguisher
UnboundLocalError: local variable 'original_address_family' referenced before assignment

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/thibault/PycharmProjects/test-g-parser/main.py", line 9, in <module>
data = parse_obj.parse_file(show_command=f'show ip bgp {address_family} vrf {vrf}', file_name_and_path='./PE_R1_show_ip_bgp_vpnv4_vrf_MPLS.txt')
File "/home/thibault/PycharmProjects/test-genie-parser/venv/lib/python3.8/site-packages/pyats_genie_command_parse/pyats_genie_command_parse.py", line 127, in parse_file
return self.__parse(show_command)
File "/home/thibault/PycharmProjects/test-genie-parser/venv/lib/python3.8/site-packages/pyats_genie_command_parse/pyats_genie_command_parse.py", line 168, in __parse
raise ModuleNotFoundError(f'Could not find module_name for command {show_command} '
ModuleNotFoundError: Could not find module_name for command show ip bgp vpnv4 vrf MPLS for nos iosxe from genie: local variable 'original_address_family' referenced before assignment

Process finished with exit code 1

Did someone saw this parser work on that command ?
Can someone investigate the reason for this issue ?

I can provide extra information if needed to clarify any point, best Regards

[Torbjørn]

I agree that this is a bug in a module(I think the parser is at fault, but I am not certain). Parsing of the output fails at the "Route Distinguisher:" line below when the ShowBgpSuperParser parser "cli" is called without the "address_family" parameter set. I unfortunately don't have the time to debug this further.

[thibaultm]

Thanks for the reply.
I understand perfectly as I work on this subject when I can, I think parsers are the most useful network automatisation tool I met.
The issue is 100% reproductible and I run my script in Pycharm so I can trace things but I am not very experienced in Python.
I can propose that when you have time you suggest me points where to put break points and then I can run again and report here the result.
The idea being that this way, step by step, we could collectively pinpoint the root cause.

And once the root cause exposed here, someone with something else to correct could include the correction with his and correct it when possible.

I am thus ready to implement your suggestion of break point when you can

[bgillies95 · Accepted solution]

I know this is a couple of months late. I ran into this issue last night also using the 'show ip bgp vpnv4 vrf <vrf-name> command.

It seems the parser is not ready to pickup the route distinguisher line in the output or the way they have built the parsing template has not accommodated for all types of output. Hopefully you were able to work it out, the same as me.

You will need to remove the line at the top of the output which states the RD and VRF specified for the BGP table.

In your case it is this line, "Route Distinguisher: 1:1 (default for vrf MPLS)"

[thibaultm]

I've just tested it works. Brilliant !
I wonder how you came to that conclusion since the software is so complex due to the fact I think that it abtracts different configuration cases (CLI, XML, etc.)

In fact what I realised through an answer in another post of the issue in github, is that it works if you use pyats (with a testbed etc.) and you query the router live (in a GNS3 simulation in my case).
it works with the regular output from the router without any modification.
This proves the issue is in pyats-genie-command-parse which is in a separate installation from pyats.

The problem is that you need a live simulation which is not always possible, it's difficult to be allowed to query production networks,
you always go through bounce servers on which you cannot install anything.

As a consequence, the possibility for pyats-genie-command-parse to parse commands on .txt files is decisive for production networks.
What is interesting is that with parsed commands it is much easier and feasible to make python scripts checking that
a migration has been successful when you have 1000+ routes etc.

In fact I am also very interested in parsing commands like :
show ip bgp vpnv4 vrf MPLS detail
I've just tested to remove the same line as above but it does not work with pyats-genie-command-parse !

It's difficult to test that command with pyats because it is implemented in recent IOS versions only.
I have a powerful PC at work and I have CML 2.7 so I can simulate the command.
But I don't see how I could send CLI commands from my ubuntu VM to the CML VM
(and thus emulating live communication with a router)
since the communication with CML VM is limited to https://
so I don't see how I could use pyats from my ubuntu VM to query the CML 2.7 VM

Thanks a lot, I hope pyats-genie-command-parse will be debugged in light of the informations you produced.
Best Regards