can't start nodes in CML

[a-gould]

Please assist me with understanding this message.  I get this same type of message for IOSv, CSR1000v, XRv and XRv9000

I have a new CML 2 install, and I'm new to this.  I created a lab, and add some nodes, but when I click the play button on a node, I get the following message.  What is causing this, and how do I fix this so that the nodes can start up?

xr9kv-0: Launch Queue (E0804 17:32:51.855988 2124 main.go:214] Disk clone failed: exit status 1 qemu-img: /var/local/virl2/images/baff40/a2502945-1ed1-4471-af7e-2fe9928c3d6b/nodedisk_0: Could not open '/var/lib/libvirt/images/virl-base-images/iosxrv9000-7-2-2/xrv9k-fullk9-x-7.2.2.qcow2': No such file or directory Could not open backing image to determine size.)

iosv-0: Launch Queue (E0804 17:36:49.459628 2174 main.go:214] Disk clone failed: exit status 1 qemu-img: /var/local/virl2/images/baff40/6df48fab-7eb2-472f-b235-dd87d97c2ab0/nodedisk_0: Could not open '/var/lib/libvirt/images/virl-base-images/iosv-159-3-m3/vios-adventerprisek9-m.spa.159-3.m3.qcow2': No such file or directory Could not open backing image to determine size.)

[balaji.bandi]

what kind of compute resources are allocated here for CML ? is there any other nodes running?

[a-gould]

12 CPU

64 GB RAM

any idea why i can't get nodes to start ?

 

[balaji.bandi]

CML 2.2 ?   if all node have same issue, suggest to reboot the CML (Linux and test )

I had the same spec I did not see the issue.  ( I am running on Esxi 6.5 as VM)

is this the first time? or is this worked before?

here is thread can help you :

https://learningnetwork.cisco.com/s/question/0D53i00000msQAw/unable-to-start-ios-xrv-9000-node

[a-gould]

yes, CML Version 2.2.1 build36

yes, all nodes (csr1kv, xrv, xrv9k, iosv), same type of error (shown below).

vm was already rebooted...twice.

is there anything i should check in the CentOS Cockpit (tcp port 9090) ?

is there anything i should check in the cml admin qui dashboard?

that thread you shared with me seems to be a different issue.

when i try to boot any node, I see this error...

csr1000v-0: Launch Queue (E0804 20:43:29.115120 3875 main.go:214] Disk clone failed: exit status 1 qemu-img: /var/local/virl2/images/baff40/78ccfc61-08f6-4851-8791-371b25d20289/nodedisk_0: Could not open '/var/lib/libvirt/images/virl-base-images/csr1000v-170302/csr1000v-universalk9.17.03.02-serial.qcow2': No such file or directory Could not open backing image to determine size.)

[a-gould]

hey look what i just found.  let me know what you think about this and if this is why i'm seeing those errors when i try to start nodes.

in this error message... it mentions...

Could not open '/var/lib/libvirt/images/virl-base-images/csr1000v-170302/csr1000v-universalk9.17.03.02-serial.qcow2': No such file or directory

when i log into the cockpit and go into "terminal" and i navigate to that directory, I don't find that qcow2 file mentioned in the above error.  i only see a yaml file.

[agould@cml2-controller csr1000v-170302]$ pwd
/var/lib/libvirt/images/virl-base-images/csr1000v-170302

[agould@cml2-controller csr1000v-170302]$ ls -la
total 8
drwxr-xr-x. 2 root root 34 Aug 4 17:30 .
drwxrwxr-x. 19 root virl2 4096 Feb 28 16:14 ..
-rw-r--r--. 1 virl2 virl2 273 Aug 4 17:30 csr1000v-170302.yaml

[balaji.bandi]

Before i go deep, is this Licensed Version ?  is this full setup done ?

as you see thre is no Images in the folder to run the Labs ( device images need to download ?)

installation guide :

https://developer.cisco.com/docs/modeling-labs/#!cml-release-notes/summary-of-changes

also check the post :

https://community.cisco.com/t5/cisco-modeling-labs-discussions/cml-2-1-2-and-vmware-player/m-p/4444614#M940

[a-gould]

yes, this is licensed.  20 node personal license.  full setup is done as far as I know.  how do i get images into the folders?  i thought the 10.5 BG refplat_p-20210511-fcs.iso file already had the images??

[balaji.bandi]

it does, there is a different ISO image here, can you post the ISO images you used to install here?

I may be missed here some information.

tell us your environment. (what CPU Intel ? or AMD ? )  ESXi ? VMware worksstation , VMware player ? bare-metal installation?

[a-gould]

My CML is running on an ESXi vSphere host running AMD Epyc processors

[a-gould · Accepted solution]

it takes a team!  myself, a linux sys engineer co-worker of mine, and the cisco tac, decided to reinstall cml 2.2.2... we all worked together and now my CML is working.  i have csr1kv and xrv nodes booting.  however, xrv9000 will boot, but the CLI/Console shows it rebooting over and over again... i'm guessing due to not enough resources.  so i will try to see what i can do with xrv.

[balaji.bandi]

Sure i was guessing some installation issue, that is the reason i was stressing and asked for installation ISO images to make sure it the correct one. Glad all is good enjoy the Labbbbbing.