9800-CL AWS command rejected

[sroic]

Hi,

We are running WLC in AWS which is deployed via Terraform. The marketplace WLC has initial stuff configured and saved in basic AMI which then gets "user data" script at boot. That way we just update the script and can rebuild the instance as much as we want. Script looks like this (and many more lines):

ios-config-101="interface GigabitEthernet1"
ios-config-102="ip address dhcp"
ios-config-103="no shut"
ios-config-104="exit"

Now, the issue we have is that some of the commands get rejected at boot up. Some of these are not even in the user data script (probably left hardcoded in the AMI somehow but not sure why they run like this). They are not important in this case but the one that runs EEM app that was supposed to pull ACLs from a separate FTP server. Applet itself is fine and working no issue when run manually, and I see in config it's loaded correctly from the user data script. So here is the log after bootup:

Jul 13 14:55:14.788: %CVAC-4-CLI_FAILURE: Configuration command failure: 'event manager run pull_acl' was rejected <-------------------------------------------------------------------???
Jul 13 14:55:14.788: %CVAC-4-CLI_FAILURE: Configuration command failure: 'exit ' was rejected
Jul 13 14:55:14.788: %CVAC-4-CLI_FAILURE: Configuration command failure: 'ip http secure-server' was rejected
Jul 13 14:55:14.788: %CVAC-4-CLI_FAILURE: Configuration command failure: 'ip http server' was rejected
Jul 13 14:55:14.788: %CVAC-4-CLI_FAILURE: Configuration command failure: 'ip http authentication local' was rejected
Jul 13 14:55:14.789: %CVAC-4-CLI_FAILURE: Configuration command failure: 'end' was rejected
Jul 13 14:55:14.789: %CVAC-3-CONFIG_ERROR: 7 error(s) while applying configs generated from file varied:/ovf-env.xml. Please inspect bootflash:/cvac.log for details.

I checked the cvac.log:

event manager run pull_acl
% (CVAC) Command failed: PRC_INVALID, PRC_FAILURE_PERMANENT
exit
% (CVAC) Command failed: PRC_INVALID, PRC_FAILURE_PERMANENT
ip http secure-server

ip http secure-server
^
% Invalid input detected at '^' marker.

% (CVAC) Command failed: PRC_PARSE_ERROR_NOMATCH, PRC_FAILURE_PERMANENT
ip http server

ip http server
^
% Invalid input detected at '^' marker.

% (CVAC) Command failed: PRC_PARSE_ERROR_NOMATCH, PRC_FAILURE_PERMANENT
ip http authentication local

ip http authentication local
^
% Invalid input detected at '^' marker.

These errors don't really help much and only "solutions" I found is that I should run those commands manually. Then the automation part makes no sense anymore. Anyone has some idea why is this happening?

[Rich R · Accepted solution]

"event manager run pull_acl" is *not* a config command it's an exec command so that's why that is failing!

Not sure why the following commands are failing - maybe it's jumping out of config mode after the error?

If you want to run an exec command while in config mode then prefix it with "do " - that will work for most commands as long as they aren't interactive (need response to prompts).

Otherwise you need something else to trigger the EEM script automatically rather than you running it manually - for example you could trigger it on the SYS-5-CONFIG_I log after you exit configuration.

[sroic]

Hi @Rich R, firstly thank you for your feedback. I've been trying to figure out this for some time and playing with different stuff around ftp, eem, escaping quotes, etc. that I totally forgot I'm not in real CLI so I tried to casually do this:

#EEM APPLET FOR ACLs
#
ios-config-1501="event manager applet pull_acl authorization bypass"
ios-config-1502="event none"
ios-config-1503="action 0001 cli command enable"
ios-config-1504="action 0002 cli command &quot;configure terminal&quot;"
ios-config-1505="action 0003 cli command &quot;file prompt quiet&quot;"
ios-config-1506="action 0004 cli command end"
ios-config-1507="action 0005 cli command &quot;copy ftp://server/file running-config&quot;"
ios-config-1508="action 0006 cli command &quot;configure terminal&quot;"
ios-config-1509="action 0007 cli command &quot;no file prompt quiet&quot;"
ios-config-1510="action 0008 cli command end"
ios-config-1511="exit"
ios-config-1512="exit"
#
ios-config-1550="event manager run pull_acl"
#

But putting "exit" twice didn't bring me in exec mode

Other rejected commands were there even before I made this applet but I didn't want to left them out of this thread just in case they share the same root cause.

After your message I was pretty confident it will work like this with "do" as you proposed:

#EEM APPLET FOR ACLs
#
ios-config-1501="event manager applet pull_acl authorization bypass"
ios-config-1502="event none"
ios-config-1503="action 0001 cli command enable"
ios-config-1504="action 0002 cli command &quot;configure terminal&quot;"
ios-config-1505="action 0003 cli command &quot;file prompt quiet&quot;"
ios-config-1506="action 0004 cli command end"
ios-config-1507="action 0005 cli command &quot;copy ftp://server/file running-config&quot;"
ios-config-1508="action 0006 cli command &quot;configure terminal&quot;"
ios-config-1509="action 0007 cli command &quot;no file prompt quiet&quot;"
ios-config-1510="action 0008 cli command end"
ios-config-1511="exit"
#
ios-config-1550="do event manager run pull_acl"

But after testing there is no change in config unfortunately, although the cvac.log doesn't throw any more issues on this command.

As you see in my code I used "no file prompt" to avoid prompts but if I don't find any more obvious reasons I will need to do some triggering with EEM.

Thanks once more!

[Rich R]

I guess you could try enable "debug ip ftp", "debug event manager all" & "debug event-manager" beforehand to try to work out why it still isn't working.

One other thing to try "do-exec" instead of "do" just in case it's treating do as ambiguous (normal CLI accepts it but maybe not like that).

[sroic]

Yeah I double checked do-exec part already, cvac.log shows it properly so should be fine, but I can update script just in case.

And will try debug (with "do" this time) and see if it helps, thx

[sroic]

Hi, just to update, "debug event manager all" showed me that my ftp request is failing for some reason. So I added a timer that waits 10 seconds after boot up and it worked. Probably something with AWS boot up process prevented using network interfaces immediately. Thanks again for the help!

[Rich R]

Excellent