EEM script didn't trigger while doing failover

[gparapitev]

Hi,

We have EEM script and IP SLA config in place but when we are trying to test the failover (wan interface goes down on active router), the script didn’t auto-triggered, and failover doesn’t happen on the backup circuit, and it got aborted as we checked on history events. Could you please suggest/help on this?

No. Job Id Proc Status Time of Event Event Type Name
1 7 Actv success Thu Oct20 14:53:11 2022 oir callback: Call Home process
2 8 Actv success Thu Oct20 14:53:14 2022 syslog callback: Call Home process
3 9 Actv abort Thu Oct20 14:56:43 2022 syslog applet: REDIRECT_ALL_TRAFFIC_TO_RTR01
4 10 Actv success Thu Oct20 14:59:07 2022 timer countdown callback: Call Home process
5 11 Actv abort Thu Oct20 15:16:37 2022 syslog applet: REDIRECT_ALL_TRAFFIC_TO_RTR01
6 12 Actv success Thu Nov17 13:00:00 2022 timer cron callback: Call Home process
7 13 Actv success Thu Nov17 13:15:00 2022 timer cron callback: Call Home process
8 14 Actv abort Tue Nov29 06:00:15 2022 track applet: INTERFACE_ALERT
9 15 Actv abort Tue Dec 6 15:04:29 2022 track applet: REDIRECT_ALL_TRAFFIC_TO_RTR02
10 16 Actv abort Tue Dec 6 15:07:59 2022 track applet: REDIRECT_ALL_TRAFFIC_TO_RTR02

[MHM Cisco World]

can I see the EEM ?

[gparapitev]

Here is the Router 1 which is active and following is the EEM script:

PUN-MTR-DIA-RTR01#sh run | sec event
event manager environment _email_from noc@mediaocean.com
event manager environment _email_to noc@mediaocean.com
event manager environment _email_server 168.238.194.7
event manager applet email-test
event none
action 8.5 mail server "$_email_server" to "$_email_to" from "$_email_from" subject "test email" body "test email received"
event manager applet REDIRECT_ALL_TRAFFIC_TO_RTR02
event track 100 state down
action 2.0 cli command "enable"
action 2.2 cli command "config t"
action 2.4 cli command "interface GigabitEthernet0/0/1"
action 2.6 cli command "standby 168 priority 90"
action 2.8 cli command "route-map PUN2TATA permit 10"
action 3.0 cli command "set as-path prepend 26057 26057 26057 26057 26057"
action 3.2 cli command "ip route 0.0.0.0 0.0.0.0 168.238.250.3 210"
action 3.4 cli command "interface loopback 100"
action 3.6 cli command "shutdown"
action 3.8 cli command "end"
action 4.0 cli command "wr mem"
action 4.2 cli command "clear ip bgp * soft"
action 8.0 syslog priority notifications msg "Failover event triggered - Pune Office Internet traffic has been redirected to the PUN-MTR-DIA-RTR02(Tata) router. Please investigate"
action 8.5 mail server "$_email_server" to "$_email_to" from "$_email_from" subject "$_event_pub_time: failover event" body "Failover event triggered - Pune Office Internet traffic has been redirected to the PUN-MTR-DIA-RTR02(Tata) router."
event manager applet RESTORE_ALL_TRAFFIC_TO_RTR01
event none
action 2.0 cli command "enable"
action 2.2 cli command "config t"
action 2.4 cli command "interface GigabitEthernet0/0/1"
action 2.6 cli command "standby 168 priority 110"
action 2.8 cli command "route-map PUN2TATA permit 10"
action 3.0 cli command "set as-path prepend 26057 26057"
action 3.2 cli command "no ip route 0.0.0.0 0.0.0.0 168.238.250.3 210"
action 3.4 cli command "interface loopback 100"
action 3.6 cli command "no shutdown"
action 3.8 cli command "end"
action 4.0 cli command "wr mem"
action 4.2 cli command "clear ip bgp * soft"
action 8.0 syslog priority notifications msg "Failback event triggered - Pune Office Internet traffic has been restored to the PUN-MTR-DIA-RTR01(Tata) router. Please confirm"
action 8.5 mail server "$_email_server" to "$_email_to" from "$_email_from" subject "$_event_pub_time: failback event" body "Failback event triggered - Pune Office Internet traffic has been restored to the PUN-MTR-DIA-RTR01(Tata) router.Please confirm"

Here is the Router 2 which is standby and following is the EEM script:

PUNPUNPUN-MTR-DIA-RTR02#sh run | sec event
event manager environment _email_from noc@mediaocean.com
event manager environment _email_to noc@mediaocean.com
event manager environment _email_server 168.238.194.7
event manager applet email-test
event event none
action 8.5 mail server "$_email_server" to "$_email_to" from "$_email_from" subject "test email" body "test email received"
event manager applet REDIRECT_ALL_TRAFFIC_TO_RTR02
event syslog pattern "HSRP-5-STATECHANGE: GigabitEthernet0/0/1 Grp 168 state Standby -> Active"
action 2.0 cli command "enable"
action 2.2 cli command "config t"
action 2.4 cli command "interface GigabitEthernet0/0/1"
action 2.6 cli command "standby 168 priority 120"
action 2.8 cli command "route-map PUN2TATA permit 10"
action 3.0 cli command "no set as-path prepend 26057 26057 26057 26057 26057"
action 3.6 cli command "end"
action 3.8 cli command "clear ip bgp * soft"
action 4.0 cli command "wr mem"
action 8.0 syslog priority notifications msg "Failover event triggered - Pune Office Internet traffic has been redirected to the PUN-MTR-DIA-RTR02(Tata) router. Please investigate"
action 8.5 mail server "$_email_server" to "$_email_to" from "$_email_from" subject "$_event_pub_time: failover event" body "Failover event triggered - Pune Office Internet traffic has been redirected to the PUN-MTR-DIA-RTR02(Tata) router."
event manager applet RESTORE_ALL_TRAFFIC_TO_RTR01
event none
action 2.0 cli command "enable"
action 2.2 cli command "config t"
action 2.4 cli command "interface GigabitEthernet0/0/1"
action 2.6 cli command "standby 168 priority 100"
action 2.8 cli command "route-map PUN2TATA permit 10"
action 3.0 cli command "set as-path prepend 26057 26057 26057 26057 26057"
action 3.6 cli command "end"
action 3.8 cli command "clear ip bgp * soft"
action 4.0 cli command "wr mem"
action 8.0 syslog priority notifications msg "Failback event triggered - Pune Office Internet traffic has been restored to the PUN-MTR-DIA-RTR01(Tata) router. Please confirm"
action 8.5 mail server "$_email_server" to "$_email_to" from "$_email_from" subject "$_event_pub_time: failover event" body "Failback event triggered-Pune Office Internet traffic has been restored to the PUN-MTR-DIA-RTR01(Tata) router.Please confirm"

[Georg Pauwen]

Hello,

post the real script you are using, not the 'event none' script (which I assume you are using for testing). Which event is supposed to trigger the EEM ?

[MHM Cisco World]

I think he talk about this 
event manager applet REDIRECT_ALL_TRAFFIC_TO_RTR02
event track 100 state down
action 2.1 cli command "enable"
action 2.2 cli command "config t"
action 2.4 cli command "interface GigabitEthernet0/0/1"
action 2.6 cli command "standby 168 priority 90"
action 2.8 cli command "route-map PUN2TATA permit 10"
action 3.0 cli command "set as-path prepend 26057 26057 26057 26057 26057"
action 3.2 cli command "ip route 0.0.0.0 0.0.0.0 168.238.250.3 210"
action 3.4 cli command "interface loopback 100"
action 3.6 cli command "shutdown"
action 3.8 cli command "end

[Georg Pauwen]

Actually, just post the script you are using, otherwise this gets confusing. Also, post the full running config (without the unused scripts).

Also, post the output of 'debug event manager action cli'

[gparapitev]

Hi,

We are getting the below output by using "debug event manager action cli" cmd.

016540: Dec 14 13:10:11.073 GMT: %TRACK-6-STATE: 100 list threshold weight Up -> Down
016541: Dec 14 13:10:11.076 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : CTL : cli_open called.
016542: Dec 14 13:10:11.178 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : OUT : PUN-MTR-DIA-RTR01>
016543: Dec 14 13:10:11.178 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : IN : PUN-MTR-DIA-RTR01>enable
016544: Dec 14 13:10:11.189 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : OUT : PUN-MTR-DIA-RTR01#
016545: Dec 14 13:10:11.189 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : IN : PUN-MTR-DIA-RTR01#config t
016546: Dec 14 13:10:21.216 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : OUT : Configuration session is locked. The lock will be cleared once you exit out of configuration mode.
016547: Dec 14 13:10:21.216 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : OUT : Enter configuration commands, one per line. End with CNTL/Z.
016548: Dec 14 13:10:21.216 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : OUT : PUN-MTR-DIA-RTR01(config)#
016549: Dec 14 13:10:21.217 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : IN : PUN-MTR-DIA-RTR01(config)#interface GigabitEthernet0/0/1
016550: Dec 14 13:10:31.136 GMT: %HA_EM-6-LOG: REDIRECT_ALL_TRAFFIC_TO_RTR02 : DEBUG(cli_lib) : : CTL : cli_close called.
016551: Dec 14 13:10:31.256 GMT:
016552: Dec 14 13:10:31.256 GMT: tty is now going through its death sequence

[MHM Cisco World]

I dont try it but 
if you enter interface mode you must exit before enter to route-map and then to global to set the static route. 

[MHM Cisco World]

I assume that you run this for HSRP group, 
I test EEM with change priority and it work excellent, 
for the as-path prepend I assume the scenario.
hope my solution is help you.