Compliance Check For Absence, URL

[tpomerhn]

Two (hopefully easy) questions on compliance reporting in NSO:

1) How can I check for compliance on something that is "not" configured, explicitly?

I have run compliance checks for, for instance, NTP servers against a device template, and it works for those NTP servers, but it doesn't object if the device has additional servers beyond those checked for, nor does it object to NTP parameters such as source-interface or ntp master, which is fine. But for some device configuration, I want to ensure that configuration is NOT there.

Classic case in point is disabling telnet from VTY lines on a router; the compliance checks I've run thus far have correctly mapped that a portion of configuration is there (i.e. transport input ssh), but how can I check that the line doesn't have "transport input all" or "transport input telnet ssh"? Is there a way to create a compliance report that says "this particular string must be exactly <x>"? Or, at least, "this part of the config cannot contain <y>"?

2) When running compliance checks, it generates a report. However, the URL given in the webgui for the report is for https://localhost:8888/<blah>, not https://nso.domain.com:8888/<blah>. Is there a way to change this URL that is printed to the WebGUI screen? I saw a similar post on this issue on DeveloperNet in the past and the response was to change the URL you're using to access the server, which isn't my problem (I'm accessing the WebGUI using the FQDN remotely, not from localhost, so the report shortcut shouldn't be using localhost).

Thanks!

[Jason Belk]

I can't speak on how to customize the compliance reports, but one way to accomplish very granular checks of you devices' configuration is to use the Python API to check if certain things are present or not. 

I created an example of this for the Python API learning lab:

https://developer.cisco.com/learning/lab/hands-on-nso-simple-python/step/1

Here is a snippet from the end of that lab :

import ncs
with ncs.maapi.single_write_trans('admin', 'python', groups=['ncsadmin']) as t:
        root = ncs.maagic.get_root(t)
        device_list = root.devices.device
        for device in device_list:
            for server in device.config.ios__radius.server:
                if '0000TYPE0PASSWORD0000' in str(server.key.secret):
                    print "for server {} old key is {}".format(server.id, server.key.secret)
                    server.key.type = "0"
                    #set new password
                    server.key.secret = "NEWPASSWORD"
                    print "new password for {} is {}".format(device.name,server.key.secret)
            #uncomment to apply the change
            #t.apply()

This snippet loops over devices, checks to see if a particular thing is present (in this case an IOS radius server key) and then does some responsive action (in this case it is changing the password, but it could have just as easily been add that non-compliant device to a list in a CSV file). 

[tpomerhn]

Thanks, appreciate the info, I am trying to do these compliance additional checks by using the CDB as a launching point for python scripts that occur off-box (I don't want to run scripts on the NSO instance, itself, because of CPU concerns). Since they're off-box and I don't know how to put the Python API on another box, I'm just using normal subprocess calls to do this, which works fine. But I'd rather see if there is a way in NSO to say "detect if this is configured, and error on it." Since it's already checking for the presence of SSH, in my previous example, it would also be nice to specify a key that I specifically DON'T want to see for compliance.

To test this, I went to a router and did "transport input ssh" - then did a sync-from, and a compliance check, and it passed. I then added "transport input ssh telnet" to a line on the router, re-sync-from'd it, and ran a compliance check, and it passed. I want that check to fail as part of the compliance check and not an external script, if possible.

I would like to think this kind of compliance check exists but I can't find it anywhere. Similar situations could present for things like DNS servers, AAA hosts, and NTP peers, all of which could be added to a router but as long as the CDB sees the ones it wants, it passes compliance. I had this happen also with NTP compliance, where three servers were added and a device passed compliance, but turns out there was another NTP server in there with "prefer" and over-rode all the ones NSO was checking against, so the compliance check was essentially worthless...

[tpomerhn]

Thanks on your #2 fix, works like a charm! I would think this would be a typical thing to set up when installing NSO but I didn't see anywhere this configuration being documented, so kudos for that fix.

As for tags, I'm okay with using them or some other template-based configuration push to validate if the configuration would change, I was just hoping that it could be integrated into the compliance module, but it appears as though that's not really an option. It's nice to be able to run compliance reports on a regular basis and get from them a list of compliant devices, but with its inability to get "negative" checks implemented, I guess I'm forced to use a two-stage compliance system (which does beg the question as to how useful the compliance module is anyways, if I have to run an external script to make up for its deficiencies - why not implement all checks there?).

Thanks again, maybe someone can have an alternative solution to my first issue in time, or maybe that's a feature in the works...

[gmuloche]

Hello,

to be clear, in my solution I am using a device-template with the tag inside my compliance report so I get the expected result inside the compliance report - the screenshot I sent you is a compliance report. So I am not sure you would need a double verification right?

FYI my compliance report config / template / ios config:

admin@ncs# show running-config compliance
compliance reports report gold-check
 compare-template gold-conf ios-grp
 !
!
admin@ncs# show running-config devices template | display tag
devices template gold-conf
 ned-id cisco-ios-cli-6.42
  config
   line vty 1 5
    ! Tags: replace (/devices/template{gold-conf}/ned-id{cisco-ios-cli-6.42:cisco-ios-cli-6.42}/config/ios:line/vty{1 5}/transport)
    transport input [ ssh ]
   !
  !
 !
!
admin@ncs# show running-config devices device ios config line
devices device ios
 config
  line vty 1 5
   transport input ssh telnet
  !
 !
!
admin@ncs# compliance reports report gold-check run
id 15
compliance-status violations
info Checking one device and no services
location http://test.host:8080/compliance-reports/report_15_admin_1_2020-3-16T15:2:22:0.xml

[tpomerhn]

Ah, sorry, I missed that part, I had assumed that you were doing more of a template-apply and seeing if it changed lines. Now I see it's part of the compliance check itself.

I'll look into that, thanks, I think that could work as well. 

[tpomerhn]

Just to be comprehensive in this question in case others search this question, yes you can also ask NSO to check for if an element explicitly does NOT exist with the tags solution suggested by gmuloche. While you can't necessarily check for "any" misconfigured element as near as I can tell (i.e. if you have three NTP servers configured, I don't see how you could match explicitly on a fourth that is wrong unless you know its IP), you can match on entire configuration commands you don't want there.

For example, on NX-OS in order to have telnet on the box as a management protocol, you need "feature telnet" in the config. To test this as something you don't want, you create a tag saying to delete it:

tpomerhn@ncs% set devices template Global_Template ned-id cisco-nx-cli-5.15 config feature telnet
tpomerhn@ncs% tag add devices template Global_Template ned-id cisco-nx-cli-5.15 config feature telnet delete
tpomerhn@ncs% commit

Now, if I run the compliance check, it fails for a device that has "feature telnet" enabled in the config, because application of the template to the device would result in changes:

Global_Template
Device nxosdevice
 config {
     feature {
-        telnet;
     }
 }

 Hope this helps in case others are looking for it.

[gmuloche · Accepted solution]

Hello,

answer for 2:

in ncs.conf you can configure: 

       /ncs-config/webui/server-name (string) [localhost]
           The hostname the Web server serves.

for instance

  <webui>
    <enabled>true</enabled>
    <server-name>test.host</server-name>
    <transport>
      <tcp>
        <enabled>false</enabled>
        <ip>0.0.0.0</ip>
        <port>8080</port>
      </tcp>
      <ssl>
        <enabled>true</enabled>
        <ip>0.0.0.0</ip>
        <port>8888</port>
        <key-file>${NCS_DIR}/var/ncs/webui/cert/host.key</key-file>
        <cert-file>${NCS_DIR}/var/ncs/webui/cert/host.cert</cert-file>
      </ssl>
    </transport>

It will change the hostname in the URL report:

admin@ncs# compliance reports report gold-check run
id 13
compliance-status no-violation
info Checking one device and no services
location https://test.host:8888/compliance-reports/report_13_admin_0_2020-3-16T14:7:41:0.xml

[gmuloche]

For 1)

a possible solution in your case is to use tags in the template:

devices template gold-conf
 ned-id cisco-ios-cli-6.42
  config
   line vty 1 5
    ! Tags: replace (/devices/template{gold-conf}/ned-id{cisco-ios-cli-6.42:cisco-ios-cli-6.42}/config/ios:line/vty{1 5}/transport)
    transport input [ ssh ]
   !
  !
 !
!

You can configue the template first and then (it can be applied on the full transport container but not on the input leaf-list at least this is what I observed):

admin@ncs(config)# tag add devices template gold-conf ned-id cisco-ios-cli-6.42 config line vty 1 5 transport replace
admin@ncs(config)# commit

You can read further on those tags in the documentation in NSO user_guide > The NSO Device Manager > Device Templates > Tags

When I gave it a try in your situation here is what I got: