cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5396
Views
21
Helpful
5
Comments
aradford
Cisco Employee
Cisco Employee

Change of focus

This blog is going to cover a work around for a current issue with "aaa command authorization" which leads to the "ERROR_HEALTH_CHECK_TIMER_EXPIRED" error for a network-device.

The fix for this is in 1.3 release of PnP, but will require IOS upgrades to support it.

Here is a work around using an Embedded Event Manager (EEM) script built into the configuration pushed to the device during Plug and Play.

I am going to assume you have seen my other blogs on PnP, so I am not going to cover the basics, just the issue and the solution.

Problem

If you have "aaa command authorization" in a configuration file you will seen the screen shot below.

errr.png

Strangely, although APIC-EM thinks the process has failed, you will see the configuration file looks correct (which it is).

Why is this happening?

  • Once you enable command authorization, the switch needs a username to authorize the commands it is running.
  • After the configuration file is downloaded to the network-device, the final step is to "write mem" and save the configuration.
  • Of course, this command will be authorized.
  • As the PnP process originates from the network-device, it has never been logged into, so does not have a valid username to authorize any exec commands. (If you look at the ISE logs, you will see a user called "async" is being used for the commands).

Solution

The solution is quite simple.  Simply leave out the "aaa command authorization" commands, and place them in an EEM script to run after the PnP process completes.

The EEM script will remove itself after it has run.

  1. The EEM script needs to also be authorized.  In this example, I have provided and user "sdn2" and a privilege level "15".  This user is not a local user, it is only defined on my tacacs server as that is where authorization is occurring.
    • event manager session cli username sdn2 privilege 15
  2. Move the "aaa" commands inside the EEM script that runs 30seconds after being created.
    • action 2.1 cli command "aaa authorization commands 1 default group ISE-T if-authenticated"
    • action 2.2 cli command "aaa authorization commands 15 default group ISE-T if-authenticated"
  3. The EEM script then removes itself.
    • action 2.3 cli command "no event manager applet POST_PNP
  4. Issue another "wr mem" to save the configuration.  This command will be authorized, so if you do not have a valid username for step #1, the write will fail.
    • action 2.9 cli command "wr mem"
  5. The two debug commands are optional.  I included them in case you want to look deeper into aaa and eem scripts.  Make sure you remove these and do "undebug all" after you go live.
    • action 1.1 cli command "debug event manager action cli"
    • action 1.2 cli command "debug aaa authorization"

hostname 3650-dns

aaa new-model

!

tacacs server ISE

address ipv4 10.10.10.130

key xxxxxx

!

aaa group server tacacs+ ISE-T

server name ISE

!

aaa authentication login default group ISE-T local

aaa authentication enable default group ISE-T enable

aaa authorization exec default group ISE-T if-authenticated

aaa accounting exec default start-stop group ISE-T

aaa accounting commands 0 default start-stop group ISE-T

aaa accounting commands 1 default start-stop group ISE-T

aaa accounting commands 15 default start-stop group ISE-T

enable password xxxxx

!

user sdn privilege 15 password xxxxx

!

ip http server

ip http secure-server

snmp-server community xxxxxx RO

!

line con 0

line vty 0 4

transport input ssh telnet

line vty 5 15

transport input ssh telnet

!

event manager session cli username sdn2 privilege 15

event manager applet POST_PNP

event timer countdown time 30

action 1.0 cli command "enable"

action 1.1 cli command "debug event manager action cli"

action 1.2 cli command "debug aaa authorization"

action 1.8 cli command "config t"

action 2.1 cli command "aaa authorization commands 1 default group ISE-T if-authenticated"

action 2.2 cli command "aaa authorization commands 15 default group ISE-T if-authenticated"

action 2.3 cli command "no event manager applet POST_PNP"

action 2.8 cli command "end"

action 2.9 cli command "wr mem"

action 3.0 cli command "end"

end

UPDATE: May 2017.  I have found situations where the EEM script fires too quickly, mainly if you are doing a management VLAN switchover, rather than pnp startup vlan.  You might need to change this to a longer countdown time.  Change from 30 to 180 seconds.

After the switch boots up, you can see that the status is successful in PnP application.

provisioned.png

Looking at the startup configuration, the "aaa authorization" commands are present.

3650-dns#show start | inc autho

aaa authorization exec default group ISE-T if-authenticated

aaa authorization commands 1 default group ISE-T if-authenticated

aaa authorization commands 15 default group ISE-T if-authenticated

What Next?

This blog covered a workaround for the "aaa command authorization" issue.  It has already been resolved in APIC-EM 1.3 and will be resolved in IOS network-device software images soon.

This solution is only for those people who would like to deploy now, on current versions of IOS.

This workaround highlights the power of EEM scripts.

In the meantime, if you would like to learn more about this, you could come hang out with us in The Cisco Devnet DNA Community. We’ll have a continuous stream of blogs like this and you can ask questions and we’ll get you answers.  In addition, we have a Github repository where you can get examples related to PnP.

Thanks for reading,

@adamradford123

5 Comments
penxu
Cisco Employee
Cisco Employee

Great info, Adam. Denali 16.3.2 or later will have the enhancement to address TACACS provisioning issue. 16.3.2 is available on CCO today.

Jacob Snyder
Level 5
Level 5

Tried to do this and had it failing even with this script.  Bumping the countdown to resolved my issue.

aradford
Cisco Employee
Cisco Employee

Yes Jacob, i ran into this myself today.  I bumped it to two mins.  Are you doing a vlan-switchover as opposed to "pnp startup-vlan"?

qm4966
Level 1
Level 1

I solved 3660X switch with eem configuraion. But i have a question.

Do you know in 2960S switch "aaa authorizaion issue..". Becauese also you know in 2960 switch plaform that don't work eem configuraion..

Henry Pinera
Cisco Employee
Cisco Employee

Can anyone share what is the minimum release needed for the 2960X platform to use in conjunction with APIC-EM 1.3+? 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: