cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
8485
Views
10
Helpful
7
Replies

ZTP IOx and GuestShell on C9300

Claudia de Luna
Spotlight
Spotlight

Hi, We are working on a proof of concept for staging C9300 switches via ZTP.

 

We tested out everything in the lab first so we knew we had a valid DHCP configuration and Python script.

 

When we unboxed some of the new switches and attempted to provision them we consistently ran into the same issue across several new switches.

They would boot up, pull a dhcp lease, pull the ztp python script and then hang.  Out of the box these C9300 were running Cisco IOS XE Software, Version 16.12.04.  We fell back to a simpler python script that just executed show commands but saw the same behavior.

 

It would hang here.

         --- System Configuration Dialog ---

Would you like to enter the initial configuration dialog? [yes/no]: 
Loading http://192.168.1.33/ztp_python_base.py 
Loading http://192.168.1.33/ztp_python_base.py 
Loading http://192.168.1.33/ztp_python_base.py 
Loading http://192.168.1.33/ztp_python_base.py 

The boot up sequence would hang here so we would then power cycle the switch: 

Initializing Hardware...

It seemed as though guestshell would never run.

 

We finally got things working by going into each switch and entering iox.

 

 

Switch#sh iox detail 

IOx Infrastructure Summary:
---------------------------
IOx service (CAF)         : Not Running
IOx service (HA)          : Not Running 
IOx service (IOxman)      : Not Running
Libvirtd                  : Running
Dockerd                   : Not Running

Switch(config)#iox 
Switch(config)#
*Jan 27 15:37:37.514: %UICFGEXP-6-SERVER_NOTIFIED_START: Switch 1 R0/0: psd: Server iox has been notified to start sh iox det
Switch(config)#do sh iox detail 

IOx Infrastructure Summary:
---------------------------
IOx service (CAF) 1.8.0.5 : Running
IOx service (HA)          : Not Running 
IOx service (IOxman)      : Not Ready 
Libvirtd                  : Running
Dockerd                   : Running

We would wait a few minutes for the Docker service to come up and then wr erase (thus removing the iox command from any configuration) and start the process again and then everything worked as expected.

 

 

         --- System Configuration Dialog ---

Would you like to enter the initial configuration dialog? [yes/no]: 
Loading http://192.168.1.33/ztp.py 
Loading http://192.168.1.33/ztp.py day0guestshell installed successfully
Current state is: DEPLOYED
day0guestshell activated successfully
Current state is: ACTIVATED
day0guestshell started successfully
Current state is: RUNNING
Guestshell enabled successfully


HTTP server statistics:
Accepted connections total: 0====== STARTING ZTP INITIALIZATION SCRIPT ======

=== Obtaining serial number of device.. ===

    --- FOC2344L33F
--- Setting configuration file variable.. ---

    --- FOC2344L33F.cfg

====== Software Version Check Cisco IOS XE Software, Version 16.12.04 ======
--- No upgrade is required!!! ---
<snip successful execution of entire python script>

 

I'm trying to understand what went wrong. I can't imaging that ZTP requires that one console into each switch and run the iox command.  I saw a similar posting ( too late) that indicated ZTP required DNS servers.  Was it as simple as that and not providing DNS servers in the DHCP lease prevented guestshell from coming up?  If so, that was not the experience in the lab (and in fact the lab DHCP server also does not set DNS servers) but in the lab we did not use a brand new out of the box switch.

 

Having to console into each switch to execute the iox command makes ZTP far less attractive so I'm hoping to ge a better understanding on what went wrong and why from this community!

 

Thanks in advance

1 Accepted Solution

Accepted Solutions

Hi @Claudia de Luna and @bigevilbeard  you may be hitting CSCvw63161 with 16.12.4 ZTP/Guestshell, there is a workaround with DHCP AutoInstall (for "low" touch) and the manual workaround is:

 

 

mkdir flash:guest-share
write erase
reload

 

 

View solution in original post

7 Replies 7

Hey @Claudia de Luna i am not sure of your issue here, will leave for others to comment - have you looked at https://developer.cisco.com/codeexchange/github/repo/tdorssers/ztp wondered if this would help here?

 

Hope this helps.

Please mark this as helpful or solution accepted to help others
Connect with me https://bigevilbeard.github.io

Hey @bigevilbeard ,  I had not seen that and I love it!  Thank you.   I think my issue is a bit more fundamental in that if Guestshell does not spin up we are dead in the water.   Not sure why we would need to console into each brand new switch and enable iox, defeats the purpose of ZTP.   I'm hoping I'm doing something dumb!  

 

In one of the Cisco Live hands on labs the guide states:

 

The device locates a DHCP server, bootstraps itself with its interface IP address, gateway, and DNS server IP address, and enables Guest Shell. The device then obtains the IP address or URL of a TFTP server and downloads a Python script to configure the device.

 

So that may very well be it. Unfortunately the 30 "new out of the box" switches all have their bootstrap configs now so I can't verify this and it does not really explain why entering the iox command would resolve this.  As of yet, I've not updated the DHCP scope to provide a DNS server.    

 

Thanks Stuart!

Hi @Claudia de Luna and @bigevilbeard  you may be hitting CSCvw63161 with 16.12.4 ZTP/Guestshell, there is a workaround with DHCP AutoInstall (for "low" touch) and the manual workaround is:

 

 

mkdir flash:guest-share
write erase
reload

 

 

Hey @jcohoe,

 

Thanks for explaining this!  Just so you know, we consoled into each switch and executed the iox command and then we did a write erase and reload.  That allowed the ZTP process to actually execute.

 

I'll take a look at DHCP Autoinstall to just do the bootstrap on the mgmt port and aaa and then do the rest via SSH.

 

Thanks again!

I ran into basically the same issue. The 9300 series switch with the 'standard ios loaded', failed after a few days of testing. 

I noticed that the script was downloaded and then immediately finishes. The guestshell wasn't loaded after DHCP and receiving the script. 

I thought/ hoped that the ZTP would do the job, but if it doesn't then the objective is difficult to achieve.


Would you like to enter the initial configuration dialog? [yes/no]:
Loading 9K/python_script.py from 10.15.255.52 (via Vlan990): !
[OK - 8646 bytes]

 

Press RETURN to get started!

Thanks @jcohoe 

Please mark this as helpful or solution accepted to help others
Connect with me https://bigevilbeard.github.io

Hi All,

 

I have been playing with this this week. I was using the script of Tim Dorssers https://github.com/tdorssers/ztp. It works like a charm now, he fixed some bugs for me. The issue is with IOS 16.12 it does not work for ZTP. Tim wrote an "auto install" version to by pass this problem. It is a TCL based script which you load via TFTP.  Anyhow for you must start the switch normally first and apply the explanation of jcohoe. (make sure the switch has no config when you reload) After that it should work.

 

So, first you need to enable DHCP on your server. The DHCP server will provide the router an IP address for the TFTP server (option 150). The DHCP server will also provide the router with the name of a script (option 67) called "kickstart-conf". The script must be in your TFTP directory. This script can be found the autoinstall directory of the ZTP-tool of Tim. If you look at the details of this script you can see that it uses eventmanger to download another TCL-script "script.tcl" make sure you move this script one level up in the directory.

 

option ip-tftp-server code 150 = { ip-address };
subnet 10.1.1.0 netmask 255.255.255.0 {
range 10.1.1.2 10.1.1.255;
option routers 10.1.1.1;
option domain-name-servers 10.1.1.1;
option ip-tftp-server 10.1.1.1;
option bootfile-name "kickstart-conf";
}

In the script.tcl you need to update the IP-address for: SYSLOG, LOGAPI and JSON to the IP of your machine. And of course you need to have a TFTP-server running. 

 

You need to be patient, it takes some time for the router to download and install the new software. Like it does when you to it manually. 

 

 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: