EEM Event is overwritting incorrect NAT statement

[westj]

I have several EEM applets that are monitoring my router for changes on the two WAN interfaces that are both DHCP enabled. Everything is almost working, but EEM is deleting a line in the config that does not exist.

Basically when an even occurs EEM is supposed to remove a NAT statement and replace it with another one pointing to different interface. But I have EEM run and delete a good line, see below:

*Aug 13 19:50:39: %TRACK-6-STATE: 110 ip sla 1 state Down -> Up
*Aug 13 19:50:39: %TRACK-6-STATE: 120 ip sla 2 state Up -> Down
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up: EEM detected track change, LTE Up
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : CTL : cli_open called.
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT : Lynx-2E>
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : IN  : Lynx-2E>en
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT : Lynx-2E#
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : IN  : Lynx-2E#conf t
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT : Enter configuration commands, one per line.  End with CNTL/Z.
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT : Lynx-2E(config)#
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : IN  : Lynx-2E(config)#no ip nat inside source list NAT-ACL interface ethernet1/2 overload
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT :
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT : Dynamic mapping in use, do you want to delete all entries? [no]:
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : IN  : yes
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT : Lynx-2E(config)#
*Aug 13 19:50:39: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : IN  : Lynx-2E(config)#ip nat inside source list NAT-ACL interface ethernet1/1 overload <--- this is the line I want, to E1/1
*Aug 13 19:50:40: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT : Lynx-2E(config)#
*Aug 13 19:50:40: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : IN  : Lynx-2E(config)#exit
*Aug 13 19:50:40: %SYS-5-CONFIG_I: Configured from console by  on vty0 (EEM:LTE-Up)
*Aug 13 19:50:40: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : OUT : Lynx-2E#
*Aug 13 19:50:40: %HA_EM-6-LOG: LTE-Up: Done with changes for LTE
*Aug 13 19:50:40: %HA_EM-6-LOG: LTE-Up : DEBUG(cli_lib) : : CTL : cli_close called.
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down: EEM detected track change, Wi-Fi Down
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : CTL : cli_open called.
*Aug 13 19:50:40:
*Aug 13 19:50:40: tty is now going through its death sequence
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : OUT : Lynx-2E>
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : IN  : Lynx-2E>en
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : OUT : Lynx-2E#
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : IN  : Lynx-2E#conf t
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : OUT : Enter configuration commands, one per line.  End with CNTL/Z.
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : OUT : Lynx-2E(config)#
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : IN  : Lynx-2E(config)#no ip nat inside source list NAT-ACL interface ethernet1/2 overload <-- it trys to delete the NAT to e1/2
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : OUT :
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : OUT : Dynamic mapping in use, do you want to delete all entries? [no]:
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : IN  : yes
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : OUT : Lynx-2E(config)#
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : IN  : Lynx-2E(config)#exit
*Aug 13 19:50:40: %SYS-5-CONFIG_I: Configured from console by  on vty1 (EEM:Wi-Fi-Down)
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : OUT : Lynx-2E#
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down: Done with changes for Wi-Fi
*Aug 13 19:50:40: %HA_EM-6-LOG: Wi-Fi-Down : DEBUG(cli_lib) : : CTL : cli_close called.
*Aug 13 19:50:40:
*Aug 13 19:50:40: tty is now going through its death sequence
*Aug 13 19:50:41: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet1/2, changed state to down
E#show run | i nat

<< no NAT line at all!! >>

How can I have EEM verify what is in the config before it deletes something?

Thanks

[Joe Clarke]

You didn't post your EEM policies.  EEM isn't going to make stuff up, so if it's executing a CLI command, then that command must be in the set of EEM actions.

[westj]

Here are my EEM configs:

event manager applet LTE-Up
 description will install the LTE NAT command
 event track 110 state up
 action 1.0 syslog msg "EEM detected track change, LTE Up"
 action 1.5 cli command "en"
 action 2.0 cli command "conf t"
 action 2.5 cli command "no ip nat inside source list NAT-ACL interface ethernet1/2 overload" pattern "no"
 action 2.8 cli command "yes"
 action 3.0 cli command "ip nat inside source list NAT-ACL interface ethernet1/1 overload"
 action 4.0 cli command "exit"
 action 6.0 syslog msg "Done with changes for LTE"
event manager applet LTE-Down
 description will remove the LTE NAT command
 event track 110 state down
 action 1.0 syslog msg "EEM detected track change, LTE Down"
 action 1.5 cli command "en"
 action 2.0 cli command "conf t"
 action 2.5 cli command "no ip nat inside source list NAT-ACL interface ethernet1/1 overload" pattern "no"
 action 2.8 cli command "yes"
 action 4.0 cli command "exit"
 action 6.0 syslog msg "Done with changes for LTE"
event manager applet Wi-Fi-Up
 description will install the Wi-Fi NAT command
 event track 120 state up
 action 1.0 syslog msg "EEM detected track change, Wi-Fi Up"
 action 1.5 cli command "en"
 action 2.0 cli command "conf t"
 action 2.5 cli command "no ip nat inside source list NAT-ACL interface ethernet1/1 overload" pattern "no"
 action 2.8 cli command "yes"
 action 3.0 cli command "ip nat inside source list NAT-ACL interface ethernet1/2 overload"
 action 4.0 cli command "exit"
 action 6.0 syslog msg "Done with changes for Wi-Fi"
event manager applet Both-Up
 description will install the Wi-Fi NAT command
 event tag 1011 track 11 state up
 event tag 1012 track 12 state up
 event tag 110 track 110 state up
 event tag 120 track 120 state up
 trigger delay 10
  correlate event 110 and event 120 and event 1011 and event 1012
 action 1.0 syslog msg "EEM detected track change, both Wi-Fi  and LTE Up, Use Wi-Fi"
 action 1.5 cli command "en"
 action 2.0 cli command "conf t"
 action 2.5 cli command "no ip nat inside source list NAT-ACL interface ethernet1/1 overload" pattern "no"
 action 2.8 cli command "yes"
 action 3.0 cli command "ip nat inside source list NAT-ACL interface ethernet1/2 overload"
 action 4.0 cli command "exit"
 action 6.0 syslog msg "Done with changes for Wi-Fi"
event manager applet Wi-Fi-Down
 description will remove  the Wi-Fi NAT command
 event track 120 state down
 action 1.0 syslog msg "EEM detected track change, Wi-Fi Down"
 action 1.5 cli command "en"
 action 2.0 cli command "conf t"
 action 2.5 cli command "no ip nat inside source list NAT-ACL interface ethernet1/2 overload" pattern "no"
 action 2.8 cli command "yes"
 action 4.0 cli command "exit"
 action 6.0 syslog msg "Done with changes for Wi-Fi"

And the SLA portion:

!
track 11 interface Ethernet1/1 line-protocol
 delay down 5 up 5
!
track 12 interface Ethernet1/2 line-protocol
 delay down 5 up 5
!
track 110 ip sla 1
!
track 120 ip sla 2

!

!
ip sla 1
 icmp-echo 4.2.2.2 source-interface Ethernet1/1
 tag LTE interface
 frequency 5
ip sla schedule 1 life forever start-time now
ip sla 2
 icmp-echo 4.2.2.2 source-interface Ethernet1/2
 frequency 5
ip sla schedule 2 life forever start-time now

Thank you

[Joe Clarke]

What's happening is that track 120 is going down, and that applet is kicking off to remove your NAT statement.  And my guess is that the "no" form of the command doesn't take into account the interface argument (try running it manually, and I'll bet the command will allow you to enter "no ip nat inside source list NAT-ACL").

My other guess is that only one NAT inside source list NAT-ACL command can exist in the config, so you won't have to "no" it.  But you'll want to test that.

[westj]

You are correct on all counts, the "no" command removes any line no matter that interface the line is calling and there can only be one at at time.

Do you know of any way to have EEM parse the config to see what is there and only delete the line if it is the incorrect one?

Thanks for the help.

[Joe Clarke · Accepted solution]

My point was you don't have to "no" anything.  Have one applet put in the e1/1 line and the other put in the e1/2 line.  Since there can only be one, you should never have a case where the wrong line is in there.

[westj]

Got it. I will clean this up and retest tomorrow.

Thanks!