IP SLA EEM to constantly reload router after ping failure

[Darren.Martinaglia]

Hi everyone, 

I'm in need of some assistance regarding EEM scripting. I am trying to create an EEM script that will reload a Router after an IP SLA failure. This is easily done and has been achieved but what I would like is to have the Router reload every interval while the tracked object is still down, so in essence, if the tracked object (IP SLA) is still down, reload the Router in 3 minutes. When the Router comes back up and the tracked object (IP SLA) is still down, reload it again after 3 minutes. I then have a script to remove the applet once the tracked object comes back up:

ip sla 1

 icmp-echo x.x.x.x source-interface Gig0/1

 frequency 10

ip sla schedule 1 start-time now life forever

!

track 1 ip sla 1 reachability

 delay down 60 up 120

!

event manager applet TRACK-1-STATE-DOWN

 event track 1 state down

 action 1.0 cli command "enable"

 action 1.1 cli command "conf t"

 action 1.2 cli command "event manager applet RELOAD-COUNTDOWN"

 action 1.3 cli command "event timer countdown time 60"

 action 1.4 cli command "action 1.0 cli command end"

 action 1.5 cli command "action 1.1 reload"

Then the below script will remove the applet once the tracked object comes back up:

event manager applet TRACK-1-STATE-UP

 event track 1 state up

 action 1.0 cli command "enable"

 action 1.1 cli command "conf t"

 action 1.2 cli command "no event manager applet RELOAD-COUNTDOWN"

 action 1.3 cli command "end"

So in the 'TRACK-1-STATE-DOWN' applet, how can I get the Router to reload if the IP SLA is still down? I've also been trying to add multiple commands to one applet but I cannot figure it out, such as:

event manager applet TRACK-1-STATE-DOWN

 action 1.0 cli command "enable"

 action 1.1 cli command "conf t"

 action 1.2 cli command "event manager applet RELOAD-COUNTDOWN"

 action 1.3 cli command "event timer countdown time 60"

 action 1.4 cli command "action 1.0 cli command end"

 action 1.5 cli command "action 1.1 cli command "copy run start" pattern "Destination"

 action 1.6 cli command "action 1.2 cli command "" "

 action 1.7 cli command "action 1.3 reload"

So I'm trying to create a new applet and save the configuration before reloading the device after 60 seconds but it does not seem to be working? The command action 1.5 cli command "action 1.1 cli command "copy run start" pattern "Destination" won't take. 

Thank you all!

D

[Joe Clarke]

The default state of the object is up, so the router should continue to reload while the target IP is unreachable.  Meaning, when the router boots up, the IP SLA operation will execute, timeout, and then take down the tracked object.  Then your "down" applet will run and install the countdown.

You shouldn't need to persist the countdown in the config.  But, if you do want to save the config, you'll need to look at How To Introduce Large Delays In EEM Policies .  In particular, the following environment variable is required to allow for the nested double quotes:

event manager environment q "

[Darren.Martinaglia]

Hi Joe,

Thanks so much for the reply and useful information. I really appreciate it. So, just to clarify, you could use the same environment to possibly nest further policies, such as shutting/no shutting an interface prior to reloading the Router if the tracked object still remains down? Would you perhaps have an example of this? I've tried to perform this myself, but I think I've hit a brick wall. I've attached it.

Thank you Joe.

[Joe Clarke]

I don't see too many issues here.  They should work with the possibility of a timeout in your one applet that waits 15 seconds.  You may need to increase its maxrun to 35 (the default is 20).  What issues are you seeing?

[Darren.Martinaglia]

Hi Joe, 

So after doing some more testing, it seems that the applet of 'TRACK-1-STATE-DOWN-BOUNCE' is been completely missed. The interface is not bounced, but rather the reload takes place first. Although I can see the configuration there for the applet of 'BOUNCE-INTERFACE' which falls under the main applet of 'TRACK-1-STATE-DOWN-BOUNCE', no action is taken at all for the interface, like it's completely bypassed?

[Joe Clarke]

Ah, that's because you can't execute a tack policy once the object is already down.  In your nested BOUNCE-INTERFACE policy, you're reacting to object 1 being down.  But it's already down.  You can only react to changes in the object state.

Therefore, change the event type of this nested applet to be a countdown timer like you did for your reload action (but obviously a smaller countdown).

[Darren.Martinaglia]

Hi Joe,

Apologies for the late reply. I've been doing some testing. I have an applet to reset an interface and if the tracked object is still down, to reload the Router. For some reason the interface reset applet fires up normally and executes successfully but the router-reload applet does not execute the reload command? When I debug the CLI (also in the attachment), I can see the router-reload applet write the configuration but then it does not execute the reload command? I've even tried quoting it with $q reload$q" but I got an error message. This Router is running eem version 4 and it's an IR809G-LTE-GA-K9 Router. When I test this on GNS3 using the same configuration, it works a treat.

Thanks Joe,

Darren

[Joe Clarke]

As I said previously, if you're going to introduce this much wait into your policies, you need a higher maxrun time.  In your reload policy, you wait 20 seconds, which is the default maxrun.  So the policy will absolutely die after the wait ends.  Increase its maxrun to at least 40.  For the reset policy where you wait 25 seconds, same story (even worse).  Increase that maxrun to at least 45.