Solved: Re: cmlutils installation not working on ubuntu 22.04 : cml --help fai

thibaultm · ‎07-11-2024

Hello I installed cmlutils on ubuntu 22.04 and even the simplest command :
cml --help
produces a traceback :

thibault@NetAutoU2204:~/pyats$ cml --help
Traceback (most recent call last):
File "/home/thibault/.local/bin/cml", line 5, in <module>
from virl.cli.main import virl
File "/home/thibault/.local/lib/python3.10/site-packages/virl/cli/main.py", line 18, in <module>
from .groups import groups # noqa: F401
File "/home/thibault/.local/lib/python3.10/site-packages/virl/cli/groups/__init__.py", line 3, in <module>
from virl.cli.groups.create.commands import create_groups
ModuleNotFoundError: No module named 'virl.cli.groups.create'
thibault@NetAutoU2204:~/pyats$

I can provide the logs of installation of cmlutils :
thibault@NetAutoU2204:~$ pip install cmlutils
Defaulting to user installation because normal site-packages is not writeable
Collecting cmlutils
Downloading cmlutils-2.2.0-py3-none-any.whl (110 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.0/111.0 KB 866.3 kB/s eta 0:00:00
Collecting virl2-client>=2.2.1
Downloading virl2_client-2.7.0-py3-none-any.whl (92 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.6/92.6 KB 1.3 MB/s eta 0:00:00
Collecting tabulate
Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)
Requirement already satisfied: PyYAML in /usr/lib/python3/dist-packages (from cmlutils) (5.4.1)
Requirement already satisfied: jinja2 in ./.local/lib/python3.10/site-packages (from cmlutils) (3.1.4)
Requirement already satisfied: requests in ./.local/lib/python3.10/site-packages (from cmlutils) (2.32.3)
Collecting libtmux>=0.34.0
Downloading libtmux-0.37.0-py3-none-any.whl (56 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.2/56.2 KB 4.6 MB/s eta 0:00:00
Requirement already satisfied: click in /usr/lib/python3/dist-packages (from cmlutils) (8.0.3)
Collecting httpx<0.27.0,>=0.26.0
Downloading httpx-0.26.0-py3-none-any.whl (75 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.9/75.9 KB 963.1 kB/s eta 0:00:00
Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib/python3/dist-packages (from jinja2->cmlutils) (2.0.1)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->cmlutils) (3.3)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3/dist-packages (from requests->cmlutils) (1.26.5)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.local/lib/python3.10/site-packages (from requests->cmlutils) (3.3.2)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->cmlutils) (2020.6.20)
Collecting anyio
Downloading anyio-4.4.0-py3-none-any.whl (86 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 86.8/86.8 KB 724.3 kB/s eta 0:00:00
Collecting sniffio
Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)
Collecting httpcore==1.*
Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 77.9/77.9 KB 912.4 kB/s eta 0:00:00
Collecting h11<0.15,>=0.13
Downloading h11-0.14.0-py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 KB 925.6 kB/s eta 0:00:00
Requirement already satisfied: typing-extensions>=4.1 in ./.local/lib/python3.10/site-packages (from anyio->httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils) (4.12.1)
Collecting exceptiongroup>=1.0.2
Downloading exceptiongroup-1.2.1-py3-none-any.whl (16 kB)
Installing collected packages: tabulate, sniffio, libtmux, h11, exceptiongroup, httpcore, anyio, httpx, virl2-client, cmlutils
Successfully installed anyio-4.4.0 cmlutils-2.2.0 exceptiongroup-1.2.1 h11-0.14.0 httpcore-1.0.5 httpx-0.26.0 libtmux-0.37.0 sniffio-1.3.1 tabulate-0.9.0 virl2-client-2.7.0
thibault@NetAutoU2204:~$

hapresto · ‎07-11-2024

I'm getting the same error, looks like a bug in cmlutils 2.2.0. I tested 2.0.1 and all is good. Checking into the bug with Joe (lead developer on cmlutils).

You can get going quick with `pip install cmlutils==2.0.1`.

View solution in original post

bigevilbeard · ‎07-11-2024

@thibaultm from your output, the error message indicates that there's a ModuleNotFoundError for virl.cli.groups.create. I guess that that the virl2-client package, which is a dependency of cmlutils, is not installed correctly or is missing some modules. For first steps, i would suggest, try reinstalling the virl2-client package, then verify that the virl2-client package is installed correctly by checking the package integrity.

If the above steps don't work, try reinstalling cmlutils

Please mark this as helpful or solution accepted to help others
Connect with me https://bigevilbeard.github.io

thibaultm · ‎07-11-2024

Hello, I did what you said, same result here are the logs :

thibault@NetAutoU2204:~$ pip3 install virl2_client
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: virl2_client in ./.local/lib/python3.10/site-packages (2.7.0)
Requirement already satisfied: httpx<0.27.0,>=0.26.0 in ./.local/lib/python3.10/site-packages (from virl2_client) (0.26.0)
Requirement already satisfied: httpcore==1.* in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2_client) (1.0.5)
Requirement already satisfied: anyio in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2_client) (4.4.0)
Requirement already satisfied: idna in /usr/lib/python3/dist-packages (from httpx<0.27.0,>=0.26.0->virl2_client) (3.3)
Requirement already satisfied: sniffio in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2_client) (1.3.1)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from httpx<0.27.0,>=0.26.0->virl2_client) (2020.6.20)
Requirement already satisfied: h11<0.15,>=0.13 in ./.local/lib/python3.10/site-packages (from httpcore==1.*->httpx<0.27.0,>=0.26.0->virl2_client) (0.14.0)
Requirement already satisfied: typing-extensions>=4.1 in ./.local/lib/python3.10/site-packages (from anyio->httpx<0.27.0,>=0.26.0->virl2_client) (4.12.1)
Requirement already satisfied: exceptiongroup>=1.0.2 in ./.local/lib/python3.10/site-packages (from anyio->httpx<0.27.0,>=0.26.0->virl2_client) (1.2.1)
thibault@NetAutoU2204:~$ cml --help
Traceback (most recent call last):
File "/home/thibault/.local/bin/cml", line 5, in <module>
from virl.cli.main import virl
File "/home/thibault/.local/lib/python3.10/site-packages/virl/cli/main.py", line 18, in <module>
from .groups import groups # noqa: F401
File "/home/thibault/.local/lib/python3.10/site-packages/virl/cli/groups/__init__.py", line 3, in <module>
from virl.cli.groups.create.commands import create_groups
ModuleNotFoundError: No module named 'virl.cli.groups.create'
thibault@NetAutoU2204:~$ pip3 install cmlutils
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: cmlutils in ./.local/lib/python3.10/site-packages (2.2.0)
Requirement already satisfied: tabulate in ./.local/lib/python3.10/site-packages (from cmlutils) (0.9.0)
Requirement already satisfied: PyYAML in /usr/lib/python3/dist-packages (from cmlutils) (5.4.1)
Requirement already satisfied: jinja2 in ./.local/lib/python3.10/site-packages (from cmlutils) (3.1.4)
Requirement already satisfied: virl2-client>=2.2.1 in ./.local/lib/python3.10/site-packages (from cmlutils) (2.7.0)
Requirement already satisfied: click in /usr/lib/python3/dist-packages (from cmlutils) (8.0.3)
Requirement already satisfied: requests in ./.local/lib/python3.10/site-packages (from cmlutils) (2.32.3)
Requirement already satisfied: libtmux>=0.34.0 in ./.local/lib/python3.10/site-packages (from cmlutils) (0.37.0)
Requirement already satisfied: httpx<0.27.0,>=0.26.0 in ./.local/lib/python3.10/site-packages (from virl2-client>=2.2.1->cmlutils) (0.26.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib/python3/dist-packages (from jinja2->cmlutils) (2.0.1)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->cmlutils) (2020.6.20)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3/dist-packages (from requests->cmlutils) (1.26.5)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.local/lib/python3.10/site-packages (from requests->cmlutils) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->cmlutils) (3.3)
Requirement already satisfied: httpcore==1.* in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils) (1.0.5)
Requirement already satisfied: anyio in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils) (4.4.0)
Requirement already satisfied: sniffio in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils) (1.3.1)
Requirement already satisfied: h11<0.15,>=0.13 in ./.local/lib/python3.10/site-packages (from httpcore==1.*->httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils) (0.14.0)
Requirement already satisfied: exceptiongroup>=1.0.2 in ./.local/lib/python3.10/site-packages (from anyio->httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils) (1.2.1)
Requirement already satisfied: typing-extensions>=4.1 in ./.local/lib/python3.10/site-packages (from anyio->httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils) (4.12.1)
thibault@NetAutoU2204:~$ cml --help
Traceback (most recent call last):
File "/home/thibault/.local/bin/cml", line 5, in <module>
from virl.cli.main import virl
File "/home/thibault/.local/lib/python3.10/site-packages/virl/cli/main.py", line 18, in <module>
from .groups import groups # noqa: F401
File "/home/thibault/.local/lib/python3.10/site-packages/virl/cli/groups/__init__.py", line 3, in <module>
from virl.cli.groups.create.commands import create_groups
ModuleNotFoundError: No module named 'virl.cli.groups.create'
thibault@NetAutoU2204:~$

Torbjørn · ‎07-11-2024

Hmm, I am unable to replicate this on my Ubuntu 22.04 machine. Could you see if you get the same result in a venv?

git clone https://github.com/CiscoDevNet/virlutils
cd virlutils
python3 -m pip venv .venv
source .venv/bin/activate
python3 -m install cmlutils
cml --help

Happy to help! Please mark as helpful/solution if applicable.
Get in touch: https://torbjorn.dev

thibaultm · ‎07-11-2024

Hello before reporting the result, I have to tell you that I have the same résult
on 2 ubuntu22.04 installations 1 is a VM, the other is on a laptop.
For information, both have pyats full installed

So I tried but I hit an issue, I tried to repair my I'm not too acquainted to venv :
thibault@NetAutoU2204:~$ git clone https://github.com/CiscoDevNet/virlutils
Clonage dans 'virlutils'...
remote: Enumerating objects: 4402, done.
remote: Counting objects: 100% (1550/1550), done.
remote: Compressing objects: 100% (698/698), done.
remote: Total 4402 (delta 851), reused 1430 (delta 796), pack-reused 2852
Réception d'objets: 100% (4402/4402), 5.09 Mio | 1.17 Mio/s, fait.
Résolution des deltas: 100% (2504/2504), fait.
thibault@NetAutoU2204:~$ ls
Bureau GNS3 iniparser Modèles Public pyats snap Vidéos
Documents Images licIOUmb-py3.py Musique py PycharmProjects Téléchargements virlutils
thibault@NetAutoU2204:~$ cd virlutils/
thibault@NetAutoU2204:~/virlutils$ python3 -m pip venv .venv
ERROR: unknown command "venv"
thibault@NetAutoU2204:~/virlutils$ python3 -m venv .venv
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

apt install python3.10-venv

You may need to use sudo with that command. After installing the python3-venv
package, recreate your virtual environment.

Failing command: /home/thibault/virlutils/.venv/bin/python3

thibault@NetAutoU2204:~/virlutils$

Now I've installed :
apt install python3.10-venv
and started again :
python3 -m pip venv .venv
but then I got the same error :
ERROR: unknown command "venv"

bigevilbeard · ‎07-11-2024

Looks like getting an error because ensurepip is not available.

Please mark this as helpful or solution accepted to help others
Connect with me https://bigevilbeard.github.io

hapresto · ‎07-11-2024

I'm getting the same error, looks like a bug in cmlutils 2.2.0. I tested 2.0.1 and all is good. Checking into the bug with Joe (lead developer on cmlutils).

You can get going quick with `pip install cmlutils==2.0.1`.

thibaultm · ‎07-11-2024

Great, it works ! thanks a lot

thibault@NetAutoU2204:~$ pip install cmlutils==2.0.1
Defaulting to user installation because normal site-packages is not writeable
Collecting cmlutils==2.0.1
Downloading cmlutils-2.0.1-py3-none-any.whl (101 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.0/102.0 KB 525.6 kB/s eta 0:00:00
Requirement already satisfied: tabulate in ./.local/lib/python3.10/site-packages (from cmlutils==2.0.1) (0.9.0)
Requirement already satisfied: libtmux>=0.34.0 in ./.local/lib/python3.10/site-packages (from cmlutils==2.0.1) (0.37.0)
Requirement already satisfied: PyYAML in /usr/lib/python3/dist-packages (from cmlutils==2.0.1) (5.4.1)
Requirement already satisfied: jinja2 in ./.local/lib/python3.10/site-packages (from cmlutils==2.0.1) (3.1.4)
Requirement already satisfied: click in /usr/lib/python3/dist-packages (from cmlutils==2.0.1) (8.0.3)
Requirement already satisfied: virl2-client>=2.2.1 in ./.local/lib/python3.10/site-packages (from cmlutils==2.0.1) (2.7.0)
Requirement already satisfied: requests in ./.local/lib/python3.10/site-packages (from cmlutils==2.0.1) (2.32.3)
Requirement already satisfied: httpx<0.27.0,>=0.26.0 in ./.local/lib/python3.10/site-packages (from virl2-client>=2.2.1->cmlutils==2.0.1) (0.26.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib/python3/dist-packages (from jinja2->cmlutils==2.0.1) (2.0.1)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->cmlutils==2.0.1) (2020.6.20)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->cmlutils==2.0.1) (3.3)
Requirement already satisfied: charset-normalizer<4,>=2 in ./.local/lib/python3.10/site-packages (from requests->cmlutils==2.0.1) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/lib/python3/dist-packages (from requests->cmlutils==2.0.1) (1.26.5)
Requirement already satisfied: httpcore==1.* in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils==2.0.1) (1.0.5)
Requirement already satisfied: anyio in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils==2.0.1) (4.4.0)
Requirement already satisfied: sniffio in ./.local/lib/python3.10/site-packages (from httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils==2.0.1) (1.3.1)
Requirement already satisfied: h11<0.15,>=0.13 in ./.local/lib/python3.10/site-packages (from httpcore==1.*->httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils==2.0.1) (0.14.0)
Requirement already satisfied: typing-extensions>=4.1 in ./.local/lib/python3.10/site-packages (from anyio->httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils==2.0.1) (4.12.1)
Requirement already satisfied: exceptiongroup>=1.0.2 in ./.local/lib/python3.10/site-packages (from anyio->httpx<0.27.0,>=0.26.0->virl2-client>=2.2.1->cmlutils==2.0.1) (1.2.1)
Installing collected packages: cmlutils
Attempting uninstall: cmlutils
Found existing installation: cmlutils 2.2.0
Uninstalling cmlutils-2.2.0:
Successfully uninstalled cmlutils-2.2.0
Successfully installed cmlutils-2.0.1
thibault@NetAutoU2204:~$ cml --help
Usage: cml [OPTIONS] COMMAND [ARGS]...

Options:
--debug / --no-debug Print any debugging output.
--help Show this message and exit.

Commands:
clear clear the current lab ID
cluster display and manage CML cluster details
cockpit opens the Cockpit UI
command send a command or config to a node (requires pyATS)
console console for node
definitions manage image and node definitions
down stop a lab
extract extract configurations from all nodes in a lab
generate generate inv file for various tools
id get the current lab title and ID
license work with product licensing
ls lists running labs and optionally those in the cache
nodes get node list for the current lab
pull pull topology.yaml from repo
rm remove a lab
save save lab to a local yaml file
search list topologies available via github
ssh ssh to a node
start start a node
stop stop a node
telnet telnet to a node
tmux console to all nodes using tmux
ui opens the Workbench for the current lab
up start a lab
use use lab launched elsewhere
version version information
wipe wipe a lab or nodes within a lab
thibault@NetAutoU2204:~$

thibaultm · ‎07-11-2024

I works then I wanted to use right away it to produce a pyats testbed file out of my CML lab.yaml file but I got the error
"Current lab is not set" though I have a .virlrc file in the same directory and the CML lab is running concurrently in another VM.
Here are the logs :

thibault@NetAutoU2204:~/Téléchargements$ cml generate pyats -o mpls_vpn_gns3_CML_IOSv_oob.yaml
Current lab is not set
thibault@NetAutoU2204:~/Téléchargements$

thibault@NetAutoU2204:~/Téléchargements$ ls -la
total 160
drwxr-xr-x 3 thibault thibault 4096 juil. 11 12:05 .
drwxr-x--- 23 thibault thibault 4096 juil. 11 16:06 ..
-rw-rw-r-- 1 thibault thibault 81298 juin 5 16:48 install_gns3.txt
-rw-rw-r-- 1 thibault thibault 60975 juin 6 14:52 install_pyATS-Full_ppyats-genie-command-parse.txt
-rwxr-xr-x 1 thibault thibault 2277 juil. 10 11:33 mpls_vpn_gns3_CML_IOSv_oob.yaml
-rw-rw-r-- 1 thibault thibault 92 juil. 11 12:05 .virlrc
thibault@NetAutoU2204:~/Téléchargements$

hapresto · ‎07-11-2024

@thibaultm Joe's a bug squashing maniac. 2.2.1 is on pypi and working.

`pip install cmlutils==2.2.1`

Torbjørn · ‎07-11-2024

WOW, that's fast!

Happy to help! Please mark as helpful/solution if applicable.
Get in touch: https://torbjorn.dev

thibaultm · ‎07-11-2024

Impressive !

Does Joes work also on pyats-genie-command-parse ?
Because I have a couple of bugs to report, parsing of :
show ip bgp vpnv4 vrf VRF_NAME
does not work with pyats-genie-command-parse but WORKS when you have a running lab with CML
and you're using pyats parsing instead, same command, same lab :
>>> output = testbed.devices['PE-R1'].parse('show ip bgp vpnv4 vrf MPLS detail')
same thing with the command :
show ip bgp vpnv4 vrf VRF_NAME detail does not work with pyats-genie-command-parse but
WORKS fine with pyats and a running lab.

The issue is the in customer production networks we never get access with pyats live,
that's why pyats-genie-command-parse is interesting since it works on an output in a .txt file !

That's also why I want to use cmlutils because I saw at that url :
https://xrdocs.io/programmability/tutorials/pyats-series-tips-and-tricks/
it is possible to parse the output of a .txt file with pyats and a testbed file.
I tried but I got an error because of my testbed file, that's why I wanted to use cmlutils
to generate one 100% clean out of my CML .yaml file

cmlutils installation not working on ubuntu 22.04 : cml --help fails