Hi, We are working on a proof of concept for staging C9300 switches via ZTP.
We tested out everything in the lab first so we knew we had a valid DHCP configuration and Python script.
When we unboxed some of the new switches and attempted to provision them we consistently ran into the same issue across several new switches.
They would boot up, pull a dhcp lease, pull the ztp python script and then hang. Out of the box these C9300 were running Cisco IOS XE Software, Version 16.12.04. We fell back to a simpler python script that just executed show commands but saw the same behavior.
It would hang here.
--- System Configuration Dialog --- Would you like to enter the initial configuration dialog? [yes/no]: Loading http://192.168.1.33/ztp_python_base.py Loading http://192.168.1.33/ztp_python_base.py Loading http://192.168.1.33/ztp_python_base.py Loading http://192.168.1.33/ztp_python_base.py The boot up sequence would hang here so we would then power cycle the switch: Initializing Hardware...
It seemed as though guestshell would never run.
We finally got things working by going into each switch and entering iox.
Switch#sh iox detail IOx Infrastructure Summary: --------------------------- IOx service (CAF) : Not Running IOx service (HA) : Not Running IOx service (IOxman) : Not Running Libvirtd : Running Dockerd : Not Running Switch(config)#iox Switch(config)# *Jan 27 15:37:37.514: %UICFGEXP-6-SERVER_NOTIFIED_START: Switch 1 R0/0: psd: Server iox has been notified to start sh iox det Switch(config)#do sh iox detail IOx Infrastructure Summary: --------------------------- IOx service (CAF) 184.108.40.206 : Running IOx service (HA) : Not Running IOx service (IOxman) : Not Ready Libvirtd : Running Dockerd : Running
We would wait a few minutes for the Docker service to come up and then wr erase (thus removing the iox command from any configuration) and start the process again and then everything worked as expected.
--- System Configuration Dialog --- Would you like to enter the initial configuration dialog? [yes/no]: Loading http://192.168.1.33/ztp.py Loading http://192.168.1.33/ztp.py day0guestshell installed successfully Current state is: DEPLOYED day0guestshell activated successfully Current state is: ACTIVATED day0guestshell started successfully Current state is: RUNNING Guestshell enabled successfully HTTP server statistics: Accepted connections total: 0====== STARTING ZTP INITIALIZATION SCRIPT ====== === Obtaining serial number of device.. === --- FOC2344L33F --- Setting configuration file variable.. --- --- FOC2344L33F.cfg ====== Software Version Check Cisco IOS XE Software, Version 16.12.04 ====== --- No upgrade is required!!! --- <snip successful execution of entire python script>
I'm trying to understand what went wrong. I can't imaging that ZTP requires that one console into each switch and run the iox command. I saw a similar posting ( too late) that indicated ZTP required DNS servers. Was it as simple as that and not providing DNS servers in the DHCP lease prevented guestshell from coming up? If so, that was not the experience in the lab (and in fact the lab DHCP server also does not set DNS servers) but in the lab we did not use a brand new out of the box switch.
Having to console into each switch to execute the iox command makes ZTP far less attractive so I'm hoping to ge a better understanding on what went wrong and why from this community!
Thanks in advance
Solved! Go to Solution.
Hey @Claudia de Luna i am not sure of your issue here, will leave for others to comment - have you looked at https://developer.cisco.com/codeexchange/github/repo/tdorssers/ztp wondered if this would help here?
Hope this helps.
Hey @bigevilbeard , I had not seen that and I love it! Thank you. I think my issue is a bit more fundamental in that if Guestshell does not spin up we are dead in the water. Not sure why we would need to console into each brand new switch and enable iox, defeats the purpose of ZTP. I'm hoping I'm doing something dumb!
In one of the Cisco Live hands on labs the guide states:
The device locates a DHCP server, bootstraps itself with its interface IP address, gateway, and DNS server IP address, and enables Guest Shell. The device then obtains the IP address or URL of a TFTP server and downloads a Python script to configure the device.
So that may very well be it. Unfortunately the 30 "new out of the box" switches all have their bootstrap configs now so I can't verify this and it does not really explain why entering the iox command would resolve this. As of yet, I've not updated the DHCP scope to provide a DNS server.
Thanks for explaining this! Just so you know, we consoled into each switch and executed the iox command and then we did a write erase and reload. That allowed the ZTP process to actually execute.
I'll take a look at DHCP Autoinstall to just do the bootstrap on the mgmt port and aaa and then do the rest via SSH.