eem script/applet not working

[herm]

i have the following applets.  the applet test work  but not the Force-InterfaceUP.

event manager applet test
 description Wan Disconnected
 event syslog id 602304
 event syslog id 603109
 action 1 cli command "en"
 action 2 cli command "config t"
 action 3 cli command "inter giga1/2"
 action 4 cli command "shutdown"
 action 5 cli command "wr mem"
 output none
event manager applet Force-InterfaceUP
 description when vpn and Dsl are up
 event syslog id 602303 occurs 1 period 3
 event syslog id 603108 occurs 1 period 10
 action 1 cli command "en"
 action 2 cli command "config t"
 action 3 cli command "inter giga1/2"
 action 4 cli command "no shutdown"
 action 5 cli command "wr mem"
 output none
event manager applet KeepVpnUP
 event timer watchdog time 30
 action 1 cli command "ping tcp inside 192.168.10.2 80 repeat 10 source 192.168.9.6 80"
 output none

here is the sho even manager"

ciscoasa(config)# sh event manager
Last Error: Command failed @ 2017/02/06 08:20:40
Consolidated syslog range: 602303-603109

event manager applet test, hits 69, last 2017/02/06 08:20:40
  last file none
  event syslog id 602304, hits 68, last 602304 @ 2017/02/06 08:20:40
  event syslog id 603109, hits 1, last 603109 @ 2017/02/01 15:11:18
  action 1 cli command "en", hits 69, last 2017/02/06 08:20:40
  action 2 cli command "config t", hits 69, last 2017/02/06 08:20:40
  action 3 cli command "inter giga1/2", hits 69, last 2017/02/06 08:20:40
  action 4 cli command "shutdown", hits 69, last 2017/02/06 08:20:40
  action 5 cli command "wr mem", hits 69, last 2017/02/06 08:20:40

event manager applet Force-InterfaceUP, hits 62, last 2017/02/06 03:20:10
  last file none
  event syslog id 602303, hits 64, last 602303 @ 2017/02/06 03:20:07
  event syslog id 603108, hits 0
  action 1 cli command "en", hits 62, last 2017/02/06 03:20:10
  action 2 cli command "config t", hits 62, last 2017/02/06 03:20:10
  action 3 cli command "inter giga1/2", hits 62, last 2017/02/06 03:20:10
  action 4 cli command "no shutdown", hits 62, last 2017/02/06 03:20:10
  action 5 cli command "wr mem", hits 62, last 2017/02/06 03:20:10

event manager applet KeepVpnUP, hits 11564, last 2017/02/07 03:12:48
  last file none
  event watchdog 30 secs, left 0 secs, hits 11564, last 2017/02/07 03:12:48
  action 1 cli command "ping tcp inside 192.168.10.2 80 repeat 10 source 192.168.9.6 80", hits 11564, last 2017/02/07 03:13:38

can someone show me why the Force-InterfaceUP doesn't kick in?

thanks,

herman

[Joe Clarke]

According to this, one of the trigger syslog messages is never generated:

event syslog id 603108, hits 0

[herm]

thank you, Joe. and yes, that is correct... i have tested on another device and this one other device do not have the same issue.  this one other device (asa5506) was able to put interface to down and up based on the syslog ids. So i copied the eem applet and copied to the one i'm having problem with but still no fix.

do you think, maybe mechanism that monitors the logs is lagging and or too many syslogs activity or the log monitor is overworking? 

[Joe Clarke]

I wouldn't think overloaded syslog on an ASA would be an issue.  But you haven't shown any proof that the syslog message in generated.  Do you see this message within the time window?

[herm]

see attached.

[Joe Clarke]

I'm not exactly sure what I should be seeing here.  It looks like you configure the applet to look for 603108, but it never gets generated in the syslog output.

[herm]

yes it didn't because the PPPoE didn't drop or disconnected.  per cisco's system log message 603108 is to PPPoE.

am more concern when the vpn tunnel goes up and sysid 602303 got generated but it didnt trigered the action 4.  per the log i uploaded, 602303 was generated but before 602304 (vpn tunnel disconnected).

event syslog id 602304, hits 68, last 602304 @ 2017/02/06 08:20:40
event syslog id 602303, hits 64, last 602303 @ 2017/02/06 03:20:07

have you seen something like this happen? and what cause it?

anyway, i've added more syslog id to trigger the action to bring the interface up. that way, the system has other syslog ids to look out for to trigger action 4 on applet event manager applet Force-InterfaceUP. so far this addition syslog id been making it work.

[Joe Clarke]

There do appear to be a couple of cases where the syslog is generated, but the applet commands fail.  I'm not sure why based on the output, and I haven't done enough with ASA EEM to tell you the debug commands to use.  I know on IOS, EEM is limited by the number of VTYs.  It could be similar on ASA if a number of EEM applets try to access the CLI at once.

[herm]

thank you Joe,

i think i here is the problem with the applets i'm playing with, the time of appearances of each syslog id does not appear in the order it supposed to be.  here is an example (the bolded time):

event manager applet Force-InterfaceDown, hits 57, last 2017/02/22 06:33:46
  last file none
  event syslog id 602304, hits 80, last 602304 @ 2017/02/22 06:32:44
  action 1 cli command "config t", hits 57, last 2017/02/22 06:33:44
  action 2 cli command "inter giga1/2", hits 57, last 2017/02/22 06:33:44
  action 3 cli command "shutdown", hits 57, last 2017/02/22 06:33:44
  action 4 cli command "wr mem", hits 57, last 2017/02/22 06:33:44
  action 5 cli command "end", hits 4, last 2017/02/22 06:33:46
  output none
event manager applet Force-InterfaceUp, hits 73, last 2017/02/22 06:33:18
  last file none
  event syslog id 602303, hits 80, last 602303 @ 2017/02/22 06:32:16
   action 1 cli command "config t", hits 73, last 2017/02/22 06:33:16
  action 2 cli command "inter giga1/2", hits 73, last 2017/02/22 06:33:16
  action 3 cli command "no shutdown", hits 73, last 2017/02/22 06:33:16
  action 4 cli command "wr mem", hits 73, last 2017/02/22 06:33:16
  action 5 cli command "end", hits 4, last 2017/02/22 06:33:18
  output none

-------------------------------------

have you ever come across such? kindly advise please...

appreciate any guidance from your side.

herman

[Joe Clarke · Accepted solution]

I have had limited EEM experience on the ASA.  I'm not sure what errors might have occurred in executing the underlying CLI commands.  If it's anything like IOS, it could be that there were not enough session resources to spawn the CLI sessions.

That said, for at least some of the times the syslog messages were generated, all commands were executed.

[herm]

thanks again, Joe.  it is working now.  i had to put a watchdog timer in between the applets to allow time for next applet to execute orderly fashion.  its been a week and applets are working how i want it to be.

thanks again.

herman