Configuring Reboot Loop on Catalyst 3850 Switches

[Daniel_B]

I recently ran into an issue at one of our remote sites where a switch won't recognize the installed fiber module. One of the ports on that module is the uplink. We suspect that the module is faulty but until we are able to get back on-site, we are reliant on a third-party to power cycle the switch in order for it to recognize the module again.

Switch   Ports          Model                  SW Version               SW Image                       Mode
------ -----           -----                  ----------              ----------                      ----
* 1         56     WS-C3850-48P             16.3.7         CAT3K_CAA-UNIVERSALK9     INSTALL

In the meantime, I created an EEM script to reboot the switch when it loses internet connectivity: 

Switch#sh run | sec ip sla
track 1 ip sla 1
delay down 60 up 120
ip sla 1
icmp-echo 8.8.8.8
frequency 10
ip sla schedule 1 life forever start-time now

Switch#sh run | sec event
event manager applet NO_CONNECTIVITY_REBOOT
event syslog pattern "1 ip sla 1 state Up -> Down" maxrun 700
trigger delay 600
action 1.0 cli command "enable"
action 2.0 cli command "reload" pattern "confirm"
action 3.0 cli command "y"

I have tested this script on a similar switch and while it does reload, it will not power cycle again until internet connectivity is restored and dropped again. From the articles I've seen, I can achieve a continuous reload cycle by setting the event to "event track 1 state down" instead of a syslog message but it seems like that option is not available on this switch.

Optimally, I would like to have the switch wait for 10 minutes before reloading and continue that cycle as long as it cannot ping 8.8.8.8. Additionally, if during that 10 minute window connectivity is restored, I would like to cancel the reload. I am very new to using EEM and would appreciate any advice. Thanks!

[christopher.uhrig]

Hi Daniel, 

regarding your limitation of the 3850 not supporting the "event track 1 state down" the following example should work for you. 

If the syslog pattern "1 ip sla 1 reachability Up -> Down" is triggered the EEM script ADD_TIMER adds the command "event timer watchdog time 600" to the EEM ISP_DOWN. This script then executes the "reload" every 10 minutes until the syslog pattern "1 ip sla 1 reachability Down -> Up" is seen and the EEM DELETE_TIMER will remove the watchdog from ISP_DOWN. 

ip sla 1
 icmp-echo 8.8.8.8
 frequency 10
ip sla schedule 1 life forever start-time now

track 1 ip sla 1 reachability
 delay down 10 up 10

event manager applet ADD_TIMER
 event syslog pattern "1 ip sla 1 reachability Up -> Down"
 action 1.0 cli command "enable"
 action 2.0 cli command "conf t"
 action 3.0 cli command "event manager applet ISP_DOWN"
 action 4.0 cli command "event timer watchdog time 600"
 action 5.0 cli command "end"
 action 6.0 cli command "write"
 action 7.0 cli command ""

event manager applet DELETE_TIMER
 event syslog pattern "1 ip sla 1 reachability Down -> Up"
 action 1.0 cli command "enable"
 action 2.0 cli command "conf t"
 action 3.0 cli command "event manager applet ISP_DOWN"
 action 4.0 cli command "no event timer watchdog time 600"
 action 5.0 cli command "end"
 action 6.0 cli command "write"
 action 7.0 cli command ""

event manager applet ISP_DOWN
 action 1.0 cli command "enable"
 action 2.0 cli command "reload" pattern "confirm"
 action 3.0 cli command ""

You can check the active watchdog timer with "show event manager statistics server" and debug the execution of the scripts with "debug event manager action cli".